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Evaluation is increasingly important for finding sustainable solutions for 
the people and the planet, based on a systematic analysis of what works, 
for whom, and under what circumstances, and to contribute to the 
achievement of the Sustainable Development Goals, as they pertain to the 
environment. 

This book explores why the Global Environment Facility (GEF) invests 
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multilateral development or environment agency places evaluation fully at 
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global environment and the changing landscape of international environ- 
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countries, multilateral development banks, UN agencies, nongovernmental 
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Further, it provides useful pointers to other organizations wishing to 
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Foreword 


Environmental matters have made it to the top of the political agenda. Public 
awareness is higher than ever, especially regarding climate change, and each 
year more public and private money is spent in an effort to restore or pre- 
serve our planet’s balance. Having worked in the field of environment for 
more than three decades and from different perspectives — academia, private 
sector, civil society, multilateral development bank, and government — 
these are encouraging developments. However, there are many questions, 
for which we all are seeking answers: are the actions commensurate? Are 
they tackling the root causes of environmental degradation? Are they lead- 
ing to transformational outcomes? Are the pioneers in the field, such as the 
Global Environment Facility (GEF), still relevant and are they using the most 
impactful approaches and instruments? One thing, though, has become clear: 
the same old same old will not suffice. Hence, whoever develops strategies or 
takes concrete investment decisions will depend even more than before on 
evidence to assure that the limited resources are well used. 

In the GEF, the evaluations by the Independent Evaluation Office (IEO), 
as well as studies by the Scientific and Technical Advisory Panel, are the 
primary sources of such evidence. Like other GEF Council members and 
other stakeholders of the “GEF family,” I am a convinced advocate and 
user of evaluations. This is especially true for the periodic Comprehensive 
Evaluations of the GEF, which draw on many thematic or geographic evalu- 
ations, involve all stakeholders, and provide detailed and timely information 
that helps enlighten the replenishment process of the GEF every four years. 
Evaluations are usually very nuanced, requiring careful reading — no simple 
recipe books. While sometimes one would wish to get simpler black/white, 
good/bad answers, it is up to the reader to draw conclusions in view of policy 
decisions and improving the strategies and designs of programs and projects. 
And rightly so, because anything else would jeopardize two characteristics of 
evaluations: the credibility and the independence. 

The three coauthors, Dr. Geeta Batra, Dr. Juha I. Uitto, and Prof. Osvaldo 
Feinstein, combine strong academic credentials with decades of practical 
experience in evaluating actions of multilateral organizations in the fields 
related to environment and sustainable development, both understood in a 
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broad sense. Hence, this book condenses a wealth of knowledge and experi- 
ence, including on new tools such as remote sensing and geospatial methods, 
which the GEF IEO has been pioneering in its evaluation work. To my 
knowledge, the GEF is the one multilateral environment or development 
organization that hardwires evaluations and decision-making more than any 
other does. This book, therefore, is recommended reading not only for eval- 
uation specialists or those involved in GEF operations. Others will also find 
relevant information, including examples from the broad range of fields that 
the GEF is dealing with, such as climate change, biodiversity, land degrada- 
tion, chemicals and waste, international waters, and forests, and the interde- 
pendence and interaction between them. These should attract great interest, 
since countless studies have highlighted that sustainable social or economic 
development requires the systematic integration of environmental aspects 
into strategy and project design. I trust that the book will find and inspire 
many interested readers. 


Dr. iur. Stefan Schwager, former GEF Council Member, representing the 
Constituency of Azerbaijan, Kazakhstan, Kyrgyz Republic, Switzerland, 
Tajikistan, Turkmenistan, and Uzbekistan. 
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1 Introduction 


Humanity is at a critical juncture of environmental, social, and economic 
factors. We know that human health and well-being are inextricably linked 
with a healthy natural environment, one with ecosystem integrity, clean air, 
and a stable climate. A stark illustration of this link is the COVID-19 pan- 
demic, with its devastating impacts on both human health and the economies 
of developing — and developed — countries around the globe. 

The SARS-CoV-2 coronavirus, which causes COVID-19, is one of the 
growing number of zoonotic viruses, which means it originated in animals 
and crossed over to humans. The increase in such viruses is directly linked 
to environmental exploitation — by encroaching into ecosystems we have 
never been exposed to before and altering interactions between animals and 
humans, we let new viruses enter our population (Berardelli 2020). 

At this juncture of humanity, independent evaluation is critical. We 
emphasize the value of independent evaluation, which means that it is carried 
out by entities and persons free from the control of those responsible for the 
design and implementation of the development intervention, and, therefore, 
objective and impartial. 

Evaluation provides valuable lessons and guidance for organizations 
and policy makers to better understand and consider the environmental- 
social-economic interlinkages in designing programs and policies. It also 
provides understanding based on evidence of what works, why, under what 
circumstances, and for whom. Evaluation answers essential questions such 
as: were the interventions the right ones and were they implemented to be 
effective? Did unintended consequences occur that were not conceived at the 
program design stage? In a world with limited resources, were the interven- 
tions designed and implemented efficiently to deliver value for money? Were 
the interventions sustainable well past implementation? 

The human-environment nexus presents additional challenges and oppor- 
tunities for evaluation. Evaluation must be able to provide evidence of how 
actions in the development domain affect the environment and vice versa. 
We need to be able to demonstrate the close interlinkages between social 
and economic development and the environment in light of evidence from 
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the real world. Evaluation must draw on scientific knowledge to complement 
analysis of real life from the field. This requires going beyond analysis of 
discrete interventions, often carried out in isolation, to assess them in the 
broader landscape and analyze how they interact with natural and human sys- 
tems. Evaluators must adopt systems-based approaches to achieve and meas- 
ure the impacts of interrelated interventions, recognizing the synergies and 
trade-offs that might exist. 

The 2030 Agenda for Sustainable Development and the Sustainable 
Development Goals (SDGs) recognize the three pillars of sustainable devel- 
opment — social, economic, and environmental — and the interrelationships 
among them. Too often, a healthy natural environment is considered an after- 
thought to human health and sustainable development as a whole, despite 
their essential interconnection. With the onset of the COVID-19 pandemic, 
a shift in attitudes toward greater care for the environment is detectable, but 
powerful interests may still seek to pursue growth at any environmental cost. 
Even in the short period of this pandemic crisis, the slowdown in economic 
activity resulted in measurable environmental and associated health benefits. 
Should we return to environmental and development business as usual after 
the crisis subsides, we will pay the price and the next pandemic will be wait- 
ing in the wings. This crisis presents an opportunity to rethink our devel- 
opment path and priorities and build back better, restructuring economies 
toward more sustainability, respect for nature, equality, and participation. 

This book draws on the findings from independent evaluation of envi- 
ronmental programs in developing countries implemented by the Global 
Environment Facility (GEF). The GEF was the first significant public fund 
established to address global environmental problems following the 1992 
Earth Summit. Although other funds dealing with climate change have since 
emerged, the GEF remains virtually the only one focusing on biodiversity 
conservation and other terrestrial and marine environmental challenges. 
Throughout the GEF portfolio, we find significant overlap between vari- 
ous environmental domains and interlinkages to socioeconomic benefits and 
human health. Across the 27 years since its founding, the GEF has weathered 
health and economic crises across the globe. Evaluations of GEF programs, 
conducted by the organization’s Independent Evaluation Office, draw on this 
long period of implementation and offer valuable evidence and insights on 
the human-environment nexus — with critical relevance for this pandemic. 
This book will also familiarize readers with approaches and current methods 
in environmental evaluation while presenting rich evidence on the impacts 
of environmental programs. 

This book’s authors are intimately involved in independent evaluation 
of the GEF and played an instrumental role in the Sixth Comprehensive 
Evaluation of the Facility.'! As independent evaluators, we are not part of GEF 
management and have a mandate to evaluate the performance, impact, and 
effectiveness of the GEF in an open manner, reporting directly to the GEF 
Council. 
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Trends in the global environment 


Fully understanding and interpreting the findings we present requires an 
appreciation of the current landscape within which the GEF is operating. 

The global environment continues to deteriorate. In fact, in a number 
of areas, damage to the global environment has exceeded critical levels and 
threatens to lead to irreversible changes in global ecosystems. The long-term 
averages of four core components of a planet suitable for human life — known 
as “planetary boundaries” — have already been exceeded: human-driven 
climate change, land system change, high levels of phosphorous and nitro- 
gen flowing into oceans due to increased fertilizer use, and loss of biosphere 
integrity (Steffen et al. 2015). All of these changes are leading the earth into 
a new state — and show no signs of slowing down. Despite efforts to achieve 
net-zero global carbon emissions by 2050, emission rates are still rising 
(World Economic Forum 2020). Several other global challenges will require 
significant public-private cooperation to address. 

The top-of-mind challenge, of course, is the global pandemic that arose in 
2020. Like the zoonotic coronavirus behind COVID-19, about two-thirds of 
all infectious diseases in humans originate in animals (Jones et al. 2008). Land 
mismanagement, habitat loss, overexploitation of wildlife, and human-induced 
climate change have created multiple pathways for pathogens to transmit from 
wildlife to domestic animals and humans. This pandemic has also revealed 
substantial vulnerabilities even in developed countries, with severe economic 
consequences. 

Another daunting challenge is the expected global population increase of 
2 billion by 2050, accompanied by a rapid increase in the global middle 
class, reaching 3 billion in the next two decades. This population growth 
accompanied by increased consumption will place unprecedented demands 
on natural resources and energy use and will result in further expansion in 
deforestation and transformation of land from nature to human uses. At the 
same time, we are seeing rapid growth in unemployment, income and wealth 
inequality within and across countries, and agrarian stress. 

The threat of human-made environmental harm was highlighted in the 
World Economic Forum’s Global Risk Report 2020, which named the top 
five global risks in terms of likelihood and severity of impact: 


1 Extreme weather events with major damage to property, infrastructure, 
and life 

2 Major biodiversity loss and ecosystem collapse (terrestrial or marine) 
with irreversible consequences for the environment 

3 Failure of climate-change mitigation and adaptation by government and 
business 

4 Major natural disasters such as earthquakes, tsunamis, volcanic eruptions, 
and geomagnetic storms 

5 Human-made environmental damage and disasters, including environ- 
mental crime, such as oil spills and radioactive contamination 
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These trends will require the world to meet increased demands for food, 
energy, human habitat, transportation, and more — all placing direct pressure 
on the global environment. 

The global landscape for environmental finance has evolved, especially 
with regard to climate finance. Today, with greater awareness of environ- 
mental issues, we have seen the launch of the SDGs, the Paris Agreement, and 
the Sendai framework for disaster risk management. In 2015, an ambitious 
plan to end poverty, protect the planet, and ensure prosperity for all was 
agreed upon unanimously by all countries: the 2030 Agenda for Sustainable 
Development (United Nations Department of Economic and Social Affairs, 
2015). It includes 17 SDGs and 169 targets reflecting the scale and ambition 
of this new universal agenda. They are integrated and indivisible and balance 
the three dimensions of sustainable development — economic, social, and 
environmental.* The Paris Agreement was the first-ever universal, legally 
binding global climate change agreement, adopted at the Paris Climate 
Conference (the 21st session of the Conference of the Parties) in December 
2015. It sets out a global framework to avoid dangerous climate change by 
limiting global warming to well below 2°C and pursuing efforts to limit it 
to 1.5°C. It also aims to strengthen countries’ ability to deal with the impacts 
of climate change and support them in their efforts.* The Sendai Framework 
for Disaster Risk Reduction 2015—2030 was the first major agreement of the 
post-2015 development agenda and provides member states with concrete 
actions to protect development gains from the risk of disaster. It works along- 
side the SDG Agenda and the Paris Agreement on Climate Change, was 
endorsed by the UN General Assembly following the 2015 Third UN World 
Conference on Disaster Risk Reduction, and advocates for “the substantial 
reduction of disaster risk and losses in lives, livelihoods and health and in the 
economic, physical, social, cultural and environmental assets of persons, 
businesses, communities and countries.” The Sendai Framework recognizes 
that the state has the primary role to reduce disaster risk but that responsibil- 
ity should be shared with other stakeholders, including local government, the 
private sector, and other stakeholders.* 

Environmental issues are being mainstreamed across a broad range of 
organizations, including the multilateral development banks. However, these 
positive developments have been affected by the international environmen- 
tal architecture of conventions, funds, programs, and donors that continues 
to show increasing fragmentation, making it more difficult to coordinate 
and harmonize funding for the implementation of environmental activities 
globally. Global funding flows in recent years have increasingly prioritized 
climate change and reduced emissions from deforestation and degradation 
(REDD+) over other environmental issues — notably biodiversity and trans- 
boundary waters. Further, the global landscape for climate change finance 
has changed significantly since the GEF become the first operating entity 
of the financial mechanism of the United Nations Framework Convention 
on Climate Change (UNFCCC) in 1996. The GEF was a principal source 
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of donor financing for climate change in the 1990s, but the landscape has 
since expanded and fragmented, and the GEF has become a relatively smaller 
contributor to climate-related projects. New multilateral institutions such 
as the Climate Investment Fund and the Green Climate Fund have pledged 
amounts that far exceed those of the GEF, and many carbon finance facili- 
ties have become active. More and more, the private sector recognizes the 
economic, financial, and human risks posed by unchecked climate change 
and has increased its capital commitments across renewable energy and other 
sectors (UN 2015). 

Despite the establishment of new funding sources and a commitment by 
the multilateral development banks, the global demand for environmen- 
tal finance to address the risks far exceeds the resources made available by 
donors. The international community has committed and invested sizable 
resources each year to address the mounting environmental issues but needs 
are huge relative to available funding and remain largely unmet. According 
to Climate Policy Initiative estimates, investment required to achieve the 
low-carbon transition ranges from $1.6 trillion to $3.8 trillion annually 
between 2016 and 2050, for supply-side energy system investments alone 
(IPCC 2018). The Global Commission on Adaptation (GCA 2019) esti- 
mates adaptation costs of $180 billion annually from 2020 to 2030. Climate 
finance flows reached a record high of $612 billion in 2017, driven by 
increased renewable energy capacity in China, the United States, and India, 
and by increased public commitments to land use and energy efficiency. 
However, 2018 saw an 11 percent drop, to $546 billion. The United Nations 
Intergovernmental Panel on Climate Change (IPCC) says that an annual 
investment of $2.4 trillion — about 2.5 percent of the world’s economy — is 
needed in the energy system alone until 2035 to limit temperature rise to 
less than 1.5°C (IPCC 2018). And the effort to tackle climate change goes 
beyond transforming energy systems: it must include spending on reforest- 
ation, coastal-defense systems, and many other efforts to cut emissions and 
adapt to rising temperatures. According to the Organisation for Economic 
Co-operation and Development, total public spending on climate finance 
was $56.7 billion in 2017 (Yeo 2019). This in turn leveraged $14.5 billion in 
private finance, for a total of $71.2 billion, well short of the IPCC funding 
target requirements. 

The response to these global environmental challenges has increased sig- 
nificantly in recent years, mainly in the area of climate change, but action 
still falls far short of the estimated funding required to meet both mitiga- 
tion and adaptation requirements. Other environmental priorities such as 
biodiversity have received even less attention. The funding situation is fur- 
ther exacerbated by the 2020 COVID-19 pandemic, increasingly frequent 
famines and natural disasters attributed to accelerating climate change that 
require immediate assistance, global market volatility, competing bilateral 
programs for environmental finance, and political uncertainty created by the 
US withdrawal from the Paris Agreement. 
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The GEF’s global role 


With its broad focus that extends to the environment beyond climate change, 
the GEF occupies a unique space in the global environmental financing archi- 
tecture and is even more important in the context of the COVID-19 pandemic. 
Despite limited funding of about $4.1 billion every four years, the GEF is the 
only institution that addresses global environmental issues rather than climate 
change alone. With a history and established standing of nearly three dec- 
ades, the GEF supports major multilateral environmental agreements beyond 
the UNFCCC, including the Convention on Biological Diversity (CBD), the 
Stockholm Convention on Persistent Organic Pollutants (POPs), the United 
Nations Convention to Combat Desertification (UNCCD), and the Minamata 
Convention on Mercury. The GEF also funds projects in the international 
waters focal area, contributing to the implementation of many global and 
regional agreements, and supports sustainable forest management initiatives 
that are consistent with the objectives of the United Nations Forum on Forests. 
GEF interventions also directly relate to SDG Goals 2, 6, 11, 13, 14, and 15 — 
on zero hunger, clean water and sanitation, sustainable cities and communities, 
climate action, life under water, and life on land. The GEF’s responses to the 
SDGs are mainly through its support to the conventions. Through its pro- 
grammatic approaches and multifocal projects, the GEF can — and does — create 
interlinkages and synergies across focal areas and planetary boundaries. 

The GEF strategy has continued to evolve to address growing environmen- 
tal challenges and the drivers of environmental degradation, with an empha- 
sis on integration. The GEF implements programs and projects in the focal 
areas of biodiversity, land degradation, climate change mitigation and adapta- 
tion, chemicals and waste, and international waters, with integrated programs 
addressing sustainable forest management, sustainable cities and food systems, 
and land use and restoration. Beginning in 2014, the GEF introduced these 
integrated approaches to assist recipient countries in meeting commitments to 
more than one global convention or thematic area by tackling underlying driv- 
ers of environmental degradation. These programs are designed to promote 
complementarities and synergies in seeking environmental benefits across var- 
ious focal areas (multiple environmental benefits). Synergy refers to multiple 
benefits that are achieved either simultaneously through a single intervention 
or through the interaction of outcomes of at least two interventions. 

The COVID-19 pandemic spotlights the GEF’s importance today. The sci- 
entific literature makes clear how destroyed habitats provide ideal conditions 
for such viruses to thrive. “We invade tropical forests and other wild landscapes, 
which harbor so many species of animals and plants — and within those crea- 
tures, so many unknown viruses,” David Quammen, author of Spillover: Animal 
Infections and the Next Pandemic, recently wrote in The New York Times. “We 
cut the trees; we kill the animals or cage them and send them to markets. We 
disrupt ecosystems, and we shake viruses loose from their natural hosts. When 
that happens, they need a new host. Often, we are it” (Quammen, 2020). 
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Figure 1.1 GEF projects by focal area 


Against this backdrop, the GEF plays a very important role in reducing 
environmental stresses, improving biodiversity, and reducing deforestation. 
Since 1992, the GEF has invested over $16.1 billion and leveraged $94.2 
billion in additional resources for more than 4,500 projects in 170 countries 
(GEF 2020). Through four funding modalities — full-size projects, medi- 
um-size projects (less than $2 million), enabling activities, and the Small 
Grants Programme — the GEF implements targeted interventions in its five 
focal areas and three integrated approach program categories. Figure 1.1 shows 
the distribution of GEF’s resources in its various focal areas — biodiversity, 
climate change, land degradation, chemicals and waste, and international 
waters, along with projects that cut across more than one focal area, referred 
to as multifocal projects.’ 


The GEF institutional framework 


The GEF was set up in 1991 as a pilot program within the World Bank. In 
1992, after the Rio Earth Summit, the GEF was restructured with three initial 
implementing agencies: the World Bank, the United Nations Development 
Programme (UNDP), and the United Nations Environment Programme 
(UNEP). Over the years, the GEF structure, governance, and partnership 
framework have evolved, including a gradual and significant increase to 18 
agencies. The expansion of the partnership aimed to increase choice, access, 
and availability for numerous underserved countries, especially least devel- 
oped countries (LDCs) and small island developing states (SIDS), based on 
agency comparative advantage. The GEF continues to be the world’s largest 
funder of public projects and programs that benefit the global environment. 
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Figure 1.2 The GEF partnership 


The GEF partnership structure 


The GEF partnership is an arrangement comprising several entities, as pre- 
sented in Figure 1.2. 


Assembly 


Each of the 183 member countries, or Participants, has a representative at the 
ministerial level in the GEF Assembly. It meets every three to four years to 
review general policies; review and evaluate the GEF’s operation based on 
reports submitted to the Council; review the membership of the Facility; and 
consider, for approval by consensus, amendments to the governing instru- 
ment of the GEF. 


GEF Council 


The Council is the GEF’s main governing body and comprises 32 Members 
appointed by constituencies of GEF member countries (14 from developed 
countries, 16 from developing countries, and two from economies in tran- 
sition). Council Members rotate every three years or until the constituency 
appoints a new Member. The Council meets twice annually to develop, 
adopt, and evaluate the operational policies and programs for GEF-financed 
activities. It also reviews and approves the work program (projects submitted 
for approval), making decisions by consensus. 


Conventions 


The GEF provides funding to assist developing countries in meeting the 
objectives of international environmental conventions. The GEF serves 
as a financial mechanism to five conventions: the CBD, the UNFCCC, 
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the Stockholm Convention on POPs, the UNCCD, and the Minamata 
Convention on Mercury. The conventions provide broad strategic guidance 
to the GEF Council and Assembly. The GEF Council converts this broad 
guidance into operational criteria for GEF projects. 


The Scientific and Technical Advisory Panel 


Six internationally recognized experts in the GEF’s key areas comprise the 
Scientific and Technical Advisory Panel (STAP), which advises the GEF on 
policies, operational strategies, programs, and projects. STAP members are 
supported by a global network of experts and institutions. The STAP interacts 
with other relevant scientific and technical bodies, particularly with the sub- 
sidiary bodies of the CBD, the UNFCCC, the UNCCD, and the Stockholm 
Convention on POPs. The UNEP, which hosts the STAP Secretariat, acts as 
its liaison with the GEF. 


The Independent Evaluation Office 


The GEF Independent Evaluation Office (IEO) is the core of the GEF part- 
nership-wide evaluation function. The IEO has the central role of ensuring 
the independent evaluation function within the GEF and operates as an organ- 
izational unit independent of GEF Secretariat management or agency. The 
IEO prepares the overall evaluation policy for the GEF to be endorsed by the 
Council, sets the minimum requirements for evaluation in the GEF partnership, 
sets an evaluation agenda for approval by the Council, validates the terminal 
evaluations of projects and programs by agencies, conducts a broad spectrum of 
independent evaluations, shares evaluative evidence and lessons learned within 
and outside the GEF, and develops recommendations for the partnership to 
improve the effectiveness of GEF policies and programs. The IEO works in 
close partnership with the global evaluation community to remain on the cut- 
ting edge of emerging and innovative methodologies in environmental evalua- 
tion. It consults and collaborates with all relevant partners to foster a network of 
evaluation professionals who may add value to GEF operations and results (GEF 
TEO 2019). Chapter 2 explores the origins and the role of the IEO. 


Trustee 


The World Bank serves as the GEF Trustee, administering the GEF Trust 
Fund of contributions by donors. Among its responsibilities, the Trustee 
helps mobilize resources for the trust fund, disburses funds to GEF Agencies, 
prepares financial reports on investments and use of resources, and monitors 
application of budgetary and project funds. 


The GEF Secretariat 


The Secretariat coordinates overall implementation of GEF activities and is 
led by a Chief Executive Officer (CEO)-Chairperson, who is appointed for 
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a four-year term by the Council, renewable for one additional term. The 
Secretariat implements decisions of the Assembly and the Council. Among 
other responsibilities, it coordinates and oversees programs, ensures policies 
are implemented in consultation with the GEF Agencies, chairs interagency 
group meetings to ensure effective collaboration among the GEF Agencies, 
and coordinates with Secretariats of the conventions. 


GEF Agencies 


The Agencies are the operational arm of the GEF. They work closely with 
project proponents — government agencies, civil society organizations, and 
other stakeholders — to design, develop, and implement GEF-funded projects 
and programs. 


GEF Focal Points 


Each of the GEF member countries has designated government officials 
responsible for GEF activities who serve as the liaison with the Secretariat 
and the GEF Agencies. All of the GEF member countries have Political Focal 
Points, who focus mainly on governance, including policies and decisions, 
and relations between member countries within their constituencies. 

Recipient countries eligible for GEF support also have Operational Focal 
Points, who are responsible for operations of GEF activities within their 
countries. This includes reviewing and endorsing project proposals to ensure 
that they are in line with national priorities and strategies. 


Conflict Resolution Commissioner 


Working directly with member countries, GEF Agencies, and affected stake- 
holders, the Commissioner helps resolve disputes and address complaints 
and other issues relevant to GEF operations. The Commissioner reports 
directly to the CEO and allows the CEO to expand feedback and respond 
more quickly to issues and concerns that may arise in GEF-funded projects. 


Organization of this book 


To explore the relevance, effectiveness, and efficiency of environmental 
interventions and explore the links between environment and socioeco- 
nomic outcomes in developing countries, we draw primarily on the Sixth 
Comprehensive Evaluation of the GEF (OPS6), completed in 2017, and the 
component evaluations conducted by the GEF IEO. OPS6 presented rich 
evidence on the impacts of environmental interventions and the policies that 
support the implementation of such programs through a partnership arrange- 
ment. As noted in the description of the GEF structure, independent eval- 
uation in the GEF is not influenced by program or agency implementation 
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units. It reports directly to the Council, which approves the IEO work pro- 
gram and budget. Although the IEO engages with other constituents of 
the GEF partnership to enhance the utility of evaluation, it is financially 
and behaviorally independent in the conduct of its evaluations. As such, the 
results and the cases we present in this book have not been influenced by 
program participants. 

Chapter 2 provides a historical perspective and discusses evolution of the 
evaluation function in the GEF. It demonstrates the shift in evaluation from 
management to an independent unit and the importance of the IEO within 
the overall GEF structure. 

The evaluation process is an important determinant of the quality of 
evaluation. Development of the approach paper, stakeholder engagement 
throughout the process, and a well-developed report with evidence and clear 
communication of findings are important processes. Chapter 3 presents the 
framework and the approach of the Sixth Comprehensive Evaluation on 
which this book is based and discusses the crucial role of evaluation in the 
GEF replenishment process. 

Chapter 4 describes the methods and tools the IEO uses and explores their 
links with the evaluation questions. We demonstrate the usefulness of remote 
sensing and geospatial methods for measuring environmental impact: they 
provide reliable and cost-effective baseline information, help detect changes 
over time, and track progress toward the achievement of environmental and 
other development targets. These methods complement our other evaluation 
approaches, and we discuss the advantages of a mixed-methods approach. 

Chapter 5 presents the overall performance results of the GEF portfolio 
and various factors driving these results. We look deeper by exploring per- 
formance ratings by region and consider the attributes affecting performance: 
project design, quality of implementation and execution, and cofinancing. 
The chapter also explores the longer term impacts and sustainability of GEF 
interventions and the channels through which these are achieved. Central 
to success for impact and sustainability is the concept of broader adoption, 
which occurs when governments and other stakeholders adopt, expand, and 
build on GEF interventions, based on their initial success. 

In its 2020 Strategy, the GEF recognized that incremental environmental 
strategies alone are not sufficient given the unprecedented nature of the pres- 
sures facing the earth’s ecosystems; rather, transformative change is impera- 
tive. Chapter 6 develops a framework for evaluating transformative change 
and presents evidence from the GEF experience. 

Evaluations must be timely and of good quality to be useful. Chapter 7 dis- 
cusses the usefulness of OPS6 and its role in the replenishment process of the 
GEF. We underscore the importance of delivering an evaluation in a timely 
manner with good quality and stakeholder participation to influence future 
programming and financing. 

In the final chapter, we reflect on the results of GEF programs and pres- 
ent lessons learned from the IEO’s program of evaluations over a period of 
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five years. These lessons are relevant for evaluators, environmental practi- 
tioners, and policymakers and applicable to the current pandemic situation. 


Notes 


1. Geeta Batra and Juha Uitto managed the component evaluations that were con- 
ducted by staff and consultants of the IEO and drafted the overall comprehensive 
evaluation (OPS6). Osvaldo Feinstein is an independent consultant and was part of 
the advisory panel for OPS6. 

. https://sustainabledevelopment.un.org/post2015/transformingourworld. 

. https://unfecc.int/process-and-meetings/the-paris-agreement/the-paris-agreement. 

. https://www.undrr.org/implementing-sendai-framework/what-sendai-framework. 


. Data as of June 30, 2020. 
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2 Evaluation at the GEF 


For an organization such as the Global Environment Facility (GEF), which 
assists numerous partners across the world in addressing some of the most 
pressing concerns of our era, a thorough grasp of the outcomes and impacts 
of its work is essential in planning and making sustainable progress. The pur- 
pose of evaluation is to understand why, how, and to what extent intended 
and unintended results accrue from interventions (whether policies, strate- 
gies, programs, or projects) and their impact on various categories of stake- 
holders. Evaluation provides accountability for the achievement of results 
and institutional performance and is an important source of knowledge and 
organizational learning. 


The evolving role of evaluation in the GEF 


The GEF evaluation function has evolved significantly over the more than 
quarter century of the fund’s existence. When the Facility was established 
in 1991 to provide new and additional funding to assist developing coun- 
tries and countries with economies in transition in meeting their obliga- 
tions under the new Convention on Biological Diversity (CBD) and the 
United Nations Framework Convention on Climate Change (UNFCCC), 
the GEF operated as a pilot facility under the auspices of the World Bank. 
The World Bank also acted as the trustee for the newly established fund, 
a role that continues today. The projects were developed and managed by 
the United Nations Development Programme (UNDP), the United Nations 
Environment Programme (UNEP), and the Bank according to eligibility 
criteria. The three implementing agencies were also in charge of monitor- 
ing and evaluation (M&E) of their respective GEF-funded portfolios. The 
GEF was designed as a partnership between the three agencies, managed by 
a chairman assisted only by a GEF operations coordinator. In April 1991, 
a GEF administrator within the World Bank Environment Department 
started work. 

At the end of the three-year pilot phase, at the request of the GEF partic- 
ipants, the three agencies commissioned an independent evaluation to learn 
from the experiences during the initial period of the GEF and to guide future 
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planning (UNDP, UNEP, and World Bank 1994). At this stage, an estimated 
$939 million had been allocated from the GEF trust fund and cofinancing 
to 112 projects involving 63 countries. The projects were in the four initial 
focal areas of biodiversity conservation, climate change mitigation, inter- 
national waters, and reducing ozone-depleting substances. The evaluation 
had a unique structure that followed the structure of the GEF. Each agency 
assembled a separate evaluation team with its own coordinator, staff, and 
evaluation manager from the agency. An independent panel of experts, the 
Scientific and Technical Advisory Panel (STAP), also was established to guar- 
antee independence and completeness of coverage, and to incorporate the 
views of nongovernmental organizations (NGOs). 

The evaluation of the pilot phase recognized a lack of comprehensive M&E 
and dissemination in the GEF, noting that: 


The major missing function of the Administrator’s Office is a system- 
atic effort to capture and disseminate the experience of the pilot phase. 
Planning for this work should have been laid out at the beginning of 
the pilot phase and developed as the program progressed. Although the 
World Bank started work in February 1993 on a database for its projects, 
neither UNDP nor UNEP had done so. This work may not fall directly 
under a communications strategy but a comprehensive GEF monitoring, 
evaluation, and dissemination strategy and operations plan for the GEF as 
a whole [emphasis in the original] is essential (UNDP, UNEP, and World 
Bank 1994, 123-124). 


The evaluators further identified this deficiency as one of their main con- 
cerns. The report noted that accountability at policy, program, and project 
levels was diffuse. It recommended the establishment of a GEF Secretariat 
that was organizationally, administratively, and functionally independent of 
the implementing agencies. The pilot phase evaluation also recommended 
making the STAP independent. These recommendations were implemented 
when the GEF continued as a regular mechanism after the pilot phase. 

Recognizing the weakness identified at the pilot phase regarding M&E, 
the GEF Council in May 1995 decided that the work program would include 
“operational monitoring and evaluation, scientific and technical monitor- 
ing and evaluation, and evaluation of strategic and cross-program issues.” 
An M&E unit was established in April 1998 as part of the GEF Secretariat, 
headed by a senior M&E coordinator.* To ensure adequate independence, 
the coordinator would report directly to the Council on evaluation matters, 
while being administratively managed by the CEO of the GEF. 

In the meantime, the Council was working to strengthen independent 
evaluation as the partnership grew.? The M&E unit was converted into an 
M&E office and its first director? started his initial five-year term in 2005, 
reporting directly to the Council. The first M&E policy was approved by the 
Council through a written procedure on February 6, 2006.5 This policy took 
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the minimum standards approach for GEF-funded activities, covering project 
design, application of M&E at the project level, and requirements for terminal 
evaluations for all projects above $500,000. The policy also required concrete 
and fully budgeted M&E plans with indicators, baselines, and designated 
responsibilities for all projects. The M&E policy confirmed the independence 
of the M&E Office and its direct reporting relationship to the Council. In 
October 2013, the Council approved the change of the office’s name to the 
GEF Independent Evaluation Office (IEO). The IEO’s role was to conduct 
independent evaluations of the GEF’s performance and impact, and to set 
and validate the standards for the evaluation function within the partnership. 
The independence of IEO has been further codified in the GEF Instrument.°® 
The M&E policy was revised in 2010 to include reference to the GEF 
results-based management system, a better definition of the roles and respon- 
sibilities for the different actors in the partnership, reference to programs and 
jointly implemented projects, and emphasis on knowledge management.’ In 
2019, the M&E policies were separated, retaining the IEO as custodian of 
the evaluation policy? while placing monitoring under the purview of the 
GEF Secretariat. This decision to differentiate the two functions followed 
international best practice and is similar to the way M&E are organized in 
global development institutions such as UNDP and the World Bank. The 
evaluation policy also recognized the expanded GEF partnership, which by 
this time comprised 18 agencies, including some international environmental 
NGOs and national and regional organizations in major recipient countries. 


Evaluation architecture in the GEF 


In the GEF partnership, evaluation is a shared responsibility. The LEO is the 
custodian of the function and is in charge of independent evaluation, but 
the agencies are responsible for evaluating their own projects and programs. 
The IEO provides guidance to the agencies regarding the design and con- 
duct of the evaluations to ensure that all project and program evaluations use 
similar approaches to cover key aspects, notably the achievement of global 
environmental objectives. 

Because the IEO work program and budget are approved by the GEF 
Council and the IEO director reports directly to the Council, a feedback 
loop is built in with policymaking, a factor that is essential in ensuring use of 
evaluation. Figure 2.1 provides a schematic presentation of how evaluation is 
positioned in the GEF. 

With responsibility for evaluating the projects and programs they imple- 
ment, the GEF Agencies are organized in myriad ways. One challenge is that 
each of the GEF Agencies has a unique strategy and organizational struc- 
ture for evaluation. In some cases, such as UNEP, a dedicated evaluation 
office conducts all project evaluations. In others, such as UNDP and the 
World Bank, the independent evaluation units validate project-level evalu- 
ations conducted or commissioned by the operational units in the agencies. 
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Figure 2.1 Evaluation in the GEF 


Several other agencies have no IEO. In all cases, the agencies must ensure 
that all GEF-funded projects are subject to a terminal evaluation. 

The IEO receives the terminal evaluation reports from the agencies, either 
through their evaluation unit or, in most cases, directly from the agency GEF 
coordination unit. The IEO then validates the reports. The terminal evalua- 
tions serve as building blocks for the IEO’s own evaluations, most important 
of which is the annual performance report that draws upon the project eval- 
uations to analyze GEF performance in terms of outcomes achieved and their 
likely sustainability. The terminal evaluation reports also allow the IEO to 
evaluate the quality of the project M&E systems. 

The IEO conducts a variety of types of evaluations, ranging from perfor- 
mance and organizational evaluations to analyses of the impact of GEF-funded 
projects and programs. The unit of analysis may be a focal area or a specific 
theme (for instance, biodiversity mainstreaming or fisheries), or it may be a 
country or a strategic cluster of countries (such as least developed countries 
or small island developing states). On the policy side, the IEO evaluates the 
design and implementation of policies related to crosscutting themes, includ- 
ing gender and indigenous peoples. Process evaluations focus on institutional 
and procedural issues as they affect the GEF’s effectiveness and efficiency. 
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The IEO maintains a relationship with the evaluation offices of the agencies 
where these are in place. Apart from validation of terminal evaluation reports, 
the IEO cooperates with other evaluation units in setting evaluation standards 
and conducting joint evaluations. Although not a joint evaluation in a strict 
sense, the evaluation of the role of local benefits in global environmental pro- 
grams (GEFEO 2006) brought together participants from several GEF part- 
ners, including UNDP, the World Bank, and the GEF Secretariat. The first 
real joint evaluation focused on the GEF activity cycle, a topic that was of con- 
siderable interest to all agencies and thus involved all of them (GEFEO 2007). 
Since then, the IEO has conducted several joint evaluations, usually with one 
partner evaluation unit from an implementing agency. Joint evaluations can 
be challenging because they require additional coordination efforts; this can 
increase transaction costs or result in cost savings, such as by sharing the costs 
of surveys. Another challenge is that each of the agencies tends to see projects 
and programs from its own unique perspective. Yet, for the same reason, joint 
evaluations can also be enriching (Carugi and Bryant 2019). 

All evaluations conducted by the IEO are presented to the GEF Council 
alongside a management response prepared by the GEF Secretariat. The 
Council makes decisions on the required follow-up after considering the 
evaluation and management response together. The IEO monitors the imple- 
mentation of the decisions and reports back to the Council on an annual basis. 

The culmination of the IEO work is the Comprehensive Evaluation of the 
GEF (formerly the Overall Performance Study [OPS]) conducted every four 
years as a prelude and input to the replenishment negotiations for the GEF 
trust fund. 

In its independent assessment of the GEF, the Multilateral Organizations 
Performance Assessment Network (MOPAN) identified six strengths, two of 
which correspond to the IEO: “Strategies are responsive to wider guidance 
and informed by a comprehensive and independent evaluation” and “The 
IEO plays an important role in delivering comprehensive, independent eval- 
uations and in ensuring the quality of monitoring and evaluation systems” 
(MOPAN 2019, 39, Box 4). 


Overall Performance Studies of the GEF 


In October 1996, the GEF Council requested that the senior M&E coordina- 
tor undertake a study of the GEF’s overall performance. The M&E unit com- 
missioned a group of independent consultants to conduct the OPS, advised by 
a senior advisory panel of experienced and knowledgeable persons in equal 
numbers from the global north and south (Porter et al. 1998). However, the 
study was unable to assess the effectiveness of the M&E program, because it 
was still in early stages of implementation at the time (Porter et al. 1998, 90). 

In time, such studies were institutionalized and became the backbone of 
the evaluation work programs tied to the four-year replenishment cycle of 
the GEF. 
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The second OPS (GEF 2002) was again commissioned by the senior M&E 
coordinator, conducted by an independent team of consultants,’ and sup- 
ported by the M&E unit. The unit at that time consisted of the senior M&E 
coordinator, three M&E specialists, and support staff. OPS2 was charged 
with gauging the GEF results and impacts during the first ten years of the 
Facility’s existence, a task that had eluded the earlier evaluations. This second 
evaluation was much more ambitious than the previous ones, as the core team 
of eight and a large group of national consultants embarked on verifying the 
results on the ground, traveling to 11 recipient countries. 

Although OPS2 shed considerable light on GEF operations at the coun- 
try level, its findings and conclusions pertained mostly to processes. The 
GEF Council was becoming impatient with regard to learning about the 
actual results in terms of environmental improvements. At the same time, 
the Council had concerns about the independence of the M&E unit, which 
was an integral part of the GEF Secretariat. When the senior M&E coordi- 
nator was launching OPS3 in 2003, the Council requested that the evalu- 
ation be conducted by a group of consultants not associated with the GEF 
or its implementing agencies. The Council approved terms of reference for 
the evaluation in April 2004" and, following a bidding process, assigned 
the project to a US-based consulting firm,'? which set out to assess progress 
toward environmental results (GEF 2005). OPS3 also assessed the effects of 
GEF structures on the Facility’s performance. During OPS3, the M&E unit 
was converted into the independent M&E Office (the GEF Evaluation Office 
[GEFEO]), which later became the IEO. The evaluation recommended that 
the M&E Office must “continue to evolve the understanding of M&E within 
the GEF” and that, together with the GEF Secretariat, the M&E Office and 
implementing and executing agencies should “ensure that M&E are covered 
at all levels,” including project, portfolio, country, network, and global envi- 
ronmental management (GEF 2005, p. 57). 

Consequently, with the technical strengthening and the enhanced inde- 
pendence of the central evaluation function, future OPSs were entrusted 
to the GEFEO. OPS4 (GEFEO 2010) was the first study undertaken by the 
GEFEO with its staff in charge of conducting specific evaluations and studies 
and the director fully responsible for the results. It was also the first OPS to 
tackle the impact of completed GEF-funded projects. The study innovated 
with a ROtI (review of outcomes to impacts) evaluation framework, which 
used a theory-based approach to track project results beyond their completion 
(GEFEO 2009; Vaessen and Todd 2008). 

OPS5 took the approach further in assessing the performance, institutional 
effectiveness, and impact of the GEF (GEF IEO 2014a). Building up to OPS5, 
the IEO had begun taking a more systematic approach to the question of impact. 
The IEO conducted impact evaluations for a geographically based cluster of 
projects and programs around the South China Sea and adjacent areas (GEFEO 
2012), projects and programs related to climate change mitigation (GEF IEO 
2014b), and, later, protected areas and protected area systems (GEF IEO 2016). 
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OPS5 also made the point that in other organizations, evaluations of such a 
broad scope tended to be called “comprehensive evaluations.” 

Taking the cue from OPS5 and recognizing that these evaluations go far 
beyond the performance of the GEF — they also cover policies, impacts, insti- 
tutional effectiveness, and governance of the partnership — the next study was 
named the Sixth Comprehensive Evaluation of the GEF (GEF IEO 2018). 
What follows delves deeper into the findings and conclusions of this com- 
prehensive evaluation, the process of its development and implementation, 
and the factors that enhanced its utility and guaranteed its use by the various 
stakeholder groups. 


Notes 


1. General Requirements for a Coordinated GEF-wide Monitoring and Evaluation 
System. Council document GEF/C.04/06 May 1995. 

2. Jarle Haarstad. 

3. The initial three Implementing Agencies (UNDP, UNEP, and the World Bank) 
expanded to ten with the addition of FAO, IFAD, UNIDO, African Development 
Bank, Asian Development Bank, European Bank for Reconstruction and Devel- 
opment, and the Inter-American Development Bank. 

4. Rob D. van den Berg. 

. GEF M&E Office: Progress Report of the Director. GEF/ME/C.28/1. 

6. The GEF shall have an Assembly, a Council, and a Secretariat including an inde- 
pendent evaluation office. In accordance with paragraph 24, a STAP shall provide 
appropriate advice. Paragraph 11. Instrument for the Establishment of the Restruc- 
tured Global Environment Facility. GEF September 2019. 

7. Revision of the GEF Monitoring and Evaluation Policy. GEF/ME/C.39/06/Rev.01. 

8. The GEF Evaluation Policy. GEF/ME/C.56/02/Rev.01. 
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. The team leader was Leif Christoffersen, a former senior World Bank Officer. 

. One of the current authors, Juha Uitto, joined the unit as M&E specialist in 1999 
at a time leading up to OPS2. 

11. GEF/C.23/4. 

12. ICF. 
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3 Evaluation and the 
replenishment process 


Most Global Environment Facility (GEF) funding takes the form of grants 
to recipient countries; thus, the fund needs to be replenished periodically. 
Taking place in a highly political context, the replenishment process is an 
extended technical and financial negotiation in which evaluation has a sig- 
nificant role that has expanded greatly since the inception of the GEF almost 
three decades ago. 


The replenishment context 


The GEF provides funding to developing countries in the form of grants, 
with the exception of a small portion provided through the non-grant instru- 
ment financing window. The Facility operates through a four-year funding 
cycle, at the end of which the donors replenish the fund. Seven replenish- 
ments of the GEF have occurred to date, showing a general upward trend in 
the amounts of funding provided to the GEF, followed by a slight decline in 
later replenishments (see Figure 3.1). 

Despite this generally positive trend in funding, each replenishment process 
has been a challenging exercise taking place against the political and financial 
backdrop of the time. Economic downturns tend to shrink the pot of money 
available for international cooperation and, consequently, reduce donor gov- 
ernments’ willingness and capacity to dole out funding. Like all international 
initiatives, the GEF has also had to deal with the vagaries of trends in interna- 
tional cooperation that tend to be reflected in financing priorities. 

Although the GEF was established explicitly to bring new and additional 
money to the table, this has, in fact, proven an illusion. At the Earth Summit! 
in 1992, the GEF became the official financial mechanism to the newly 
minted Rio Conventions, with emphasis that the funding provided would 
be over and above official development assistance (ODA). The developing 
countries would be responsible for identifying projects to meet their own 
interests and ODA would support these efforts. The intended role of the GEF 
was to ensure that the development would become environmentally sound. 
This was in the global interest and the industrialized countries would be 
responsible for covering the costs. This reflected the historical responsibility 
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Figure 3.1 GEF funding by replenishment period 


of the developed countries for climate change, which led to the inclusion of 
the principle of “common but differentiated responsibilities” embedded in the 
UN Framework Convention on Climate Change.” The developing countries 
had a legitimate claim to development funding; protecting the global envi- 
ronment was seen not as part of development but as a separate objective. Some 
developing countries and civil society organizations, in fact, perceived the 
GEF as a donor instrument to dictate a Northern environmental agenda upon 
the Global South (Gupta 1995). In the donor countries of the Global North, 
however, politicians and the general public found this distinction to be rather 
academic: aid for the developing countries and countries with economies 
in transition to protect the global environment was still aid, and the money 
would most often be channeled through ODA mechanisms. Soon after the 
Earth Summit, an economic downturn diminished support for foreign aid in 
many countries and reduced the overall flows. Money channeled to the GEF 
was, in practice, taken away from somewhere else. And that was acceptable, 
because one of the purposes of the GEF was the greening of ODA. 

The GEF has always had to compete for funding with other priorities, both 
within and outside ODA. Despite the urgency and massive needs for tackling 
global environmental degradation, biodiversity conservation, harmful chem- 
icals management, and climate change, there was a limit as to how large the 
GEF pie could grow. Nominal formulas were established to determine 
the level of participating countries’ contributions in relation to how much 
the largest contributor’ put into the fund (Clémencon 2006). Particularly in 
earlier years, with perceptions of a clearer division between economic devel- 
opment and environmental protection, the GEF was seen as niche funding 
for the global environment. 
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Today the international development community better understands the 
close interlinkages between environment and development — indeed, the 
Sustainable Development Goals (SDGs) stand on three pillars: social, eco- 
nomic, and environmental. However, the GEF-8 replenishment process in 
2021-2022 will be taking place under the shadow of recovery from the global 
stress generated by the COVID-19 crisis, and the GEF will be competing for 
funding with organizations and programs dealing more directly with health. 

Within the GEF, the different environmental priorities have had to com- 
pete with each other for a funding share in what is essentially a zero-sum 
game. In early years, the allocations were made simply between the focal 
areas of biodiversity, climate change, and international waters, as well as sup- 
port to the phasing out of ozone depleting substances. As the GEF has grown 
more complex, the allocation process has become more multifaceted, but the 
key question remains: how to prioritize between the different areas and types 
of intervention within a limited financial envelope? 


Replenishment requires evaluation 


Making informed decisions about the GEF programing focus requires 
knowledge of what has proven effective in tackling the global environmental 
challenges in the organization’s purview. Systematic evaluation is one of the 
best ways of generating evidence of past performance; factors contributing 
to results (or lack thereof); and the likelihood of achieving desired outcomes 
and impacts in the most effective, efficient, and sustainable manner. Each of 
the replenishments has been preceded by a comprehensive evaluation, an 
overall performance study (OPS) as described in Chapter 2. 

The negotiations for the GEF’s seventh replenishment (GEF-7) illustrate 
the vital role of evaluation. The Sixth Comprehensive Evaluation of the GEF 
(OPS6) became one of the three key documents informing the process. The 
other two were the GEF-7 Policy Agenda and Programming Directions that 
were prepared by the GEF Secretariat — these documents initially took into 
account evaluative evidence emerging from the ongoing OPS6 and were 
modified until the very end of the replenishment process based on the replen- 
ishment participants’ deliberations, which were also informed by evaluation. 

Participants in the replenishment process are countries that commit to 
investing a minimum of $3 million into the GEF trust fund. This opens the 
door to allow developing countries to become replenishment participants 
while still remaining recipients of GEF support. In the case of GEF-7 replen- 
ishment, countries such as Brazil, China, Côte d’Ivoire, Ethiopia, India, and 
South Africa made these contributions, in addition to the traditional donors 
from the Global North. 

Of course, each of the countries and groups of countries has its own pri- 
orities that it seeks to promote for either technical or political reasons. A 
partnership like the GEF also has other interested groups advocating for their 
own interests. The most important group is the recipient countries that are 
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represented in the replenishment meetings through country constituency 
groupings. Another important stakeholder group consists of the 18 agencies 
that have direct access to GEF funding and implement GEF-funded projects 
and programs. 

A fundamental issue in replenishment pertains to country allocations. 
Proceeding entirely on the basis of where the biggest issues affecting the global 
environment can be tackled most effectively might lead to favoring large, 
megadiverse countries such as Brazil, host to the largest part of the world’s most 
significant rainforest, where environmental benefits can be achieved on mul- 
tiple fronts by addressing biodiversity conservation, deforestation, land degra- 
dation, carbon sequestration, water management, fisheries, etc. Attention also 
must be paid to countries that have fewer resources of their own, including 
least developed countries (LDCs) and small island developing states (SIDS). 
Supporting this latter point is the fact that many donors consider GEF funding 
as part of their ODA, which often focuses on reducing poverty. This has also 
led to discussions about the funding share that goes to middle-income coun- 
tries. In response, the GEF established an elaborate resource allocation system, 
originally the Resource Allocation Framework and later modified into the 
System for Transparent Allocation of Resources (STAR). 

The role of evaluation under these circumstances is to provide technical and 
scientific evidence regarding past performance, allowing the replenishment 
participants to make informed decisions regarding policies and programs 
within the current socioeconomic and political context. The evaluative evi- 
dence pertains to the achievement of impacts, organizational performance, 
and the effectiveness of institutional and governance arrangements. 


How the process incorporates evaluation 


Setting up the evaluation agenda to contribute to such a complex process 
as the GEF replenishment requires considering several important factors in 
detail. The evaluation must address all of the key aspects that have a bear- 
ing on the policies and programing directions of the next GEF phase. The 
evaluative evidence presented must be credible — and be perceived as such. 
Extensive effort and consultation go into defining the focus, scope, approach, 
and methodologies of the overall evaluation and all the component stud- 
ies that contribute to it. The analyses conducted by the GEF Independent 
Evaluation Office (IEO) must encompass as broad a set of evidence as possible 
to allow for triangulation of findings (Carugi 2016). The IEO must also be 
seen as independent and impartial in posing the evaluation questions and 
providing answers to them. 

Another key aspect of utility is timeliness. We may produce detailed evi- 
dence based on unassailable and thorough analysis, but if it comes too late, it 
will be unusable (Uitto 2016). This poses a difficult dilemma. On one hand, 
conducting the evaluations as late in the process as possible will reflect the 
latest development in programing and give outcomes time to emerge. On the 
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other, decision-makers need evaluation results as early as possible to make 
the best use of them. These issues are considered in more detail in Chapter 7. 

Our development of OPS6 started with a comprehensive approach paper 
(GEF IEO 2016) that first set the stage for the evaluation, including the envi- 
ronmental context. It further defined the scope of the evaluation, including 
the main aspects to be covered in terms of results and impact, and organi- 
zational, institutional, and governance topics. It presented the specific ques- 
tions that the evaluation would seek to answer and the approach taken. The 
approach paper also identified all the sources of evidence to be used, includ- 
ing the component evaluations and studies that would feed into the larger 
comprehensive evaluation. Finally, the approach paper provided a detailed 
timeline for the evaluation. 

We also set up an independent advisory panel consisting of five interna- 
tionally renowned experts, carefully selected for their expertise in a range of 
relevant subject matters, evaluation itself, and institutional knowledge. The 
panel also had good geographic and gender balance. Their first job was to 
review and comment on the approach paper and they played a critical role in 
finalizing the full evaluation and helping the IEO to craft the recommenda- 
tions. Their presence also added to the credibility of the evaluation. 

The IEO developed the approach paper in a consultative mode, seeking the 
inputs and feedback from key stakeholders including the GEF Secretariat and 
agencies. We circulated the approach paper for advance review and presented 
it to the GEF Council for debate and decision-making in June 2016. This was 
an important step because it allowed the Council to take ownership of the 
evaluation and to advise the IEO of additional items the Council wanted to 
emphasize. This also created the opportunity for consensus on the timeline 
for presenting findings to the replenishment process. 

We identified the key audiences for the evaluation as the Council, the 
replenishment participants (also represented within the Council), and the 
GEF Secretariat, who would need to use the emerging evaluation findings in 
developing the Policy Agenda and Programming Directions documents for 
GEF-7. Other primary audiences included the GEF Agencies and civil soci- 
ety organizations in the GEF network. Secondary audiences included inter- 
ested outside parties, such as academics focused on the global environment. 

In parallel with the preparations for OPS6, the Council agreed on the 
replenishment schedule. There would be four meetings: 


e March 2017, hosted by the World Bank as the GEF Trustee in its Paris 
office 

e October 2017, hosted by the Government of Ethiopia in Addis Ababa 

e January 2018, hosted by Government of Brazil in São Paulo 

e April 2018, hosted by the Government of Sweden in Stockholm 


To make the greatest use of the data, the Council requested that the IEO 
present early emerging findings at the Paris meeting — only nine months after 
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the approval of the approach paper. This presented a challenge to the IEO to 
focus on completing as many of the component evaluations as possible. This 
was a departure from the earlier practice of presenting a draft OPS report to 
the replenishment meetings only once all component evaluations had been 
assembled. The old practice did provide the replenishment participants with 
a complete view of GEF performance, but the inputs came late for inclusion 
in the policy and programing documents. We saw the benefits of the new 
approach and concentrated on completing as many of the component evalu- 
ations as possible within the timeline. 

By the first replenishment meeting, we had completed or nearly completed 
every component evaluation and study. Many were not in final report form 
that could be shared, so we decided to prepare a four-page note on every 
evaluation. This note explained what the evaluation was about, the approach 
taken and the portfolio and case studies covered, and the findings and con- 
clusions (also see Chapter 7). For evaluations of projects or programs in their 
early stages, we included only the emerging findings of which we could be 
quite confident. These four-page notes were shared with the replenishment 
participants at the first meeting with a lengthy presentation by the IEO direc- 
tor. The presentation led to an even longer discussion as the replenishment 
participants wanted to spend adequate time to understand the implications of 
each evaluation that was presented. 

This meeting was a very influential moment in the GEF-7 replenishment 
process. It enhanced the participants’ understanding of the issues and of the 
GEP’s strengths and weaknesses. It also allowed all of this evaluation infor- 
mation to flow into the preparation of the Policy Agenda and Programming 
Directions documents. 

In the following months, the IEO completed all of the component evalu- 
ations and compiled a full draft OPS6 report during the summer of 2017. A 
critical component of the preparation was a workshop in June 2017 with the 
advisory panel, in which the panel members collaborated with IEO manage- 
ment and staff to tease out the large themes that the evidence brought out. 
Following the workshop, IEO management and staff wrote the summary 
report that became OPS6 (GEF IEO 2017). 

The completed draft evaluation report was presented at the second replen- 
ishment meeting in Addis Ababa in October 2017, when the replenishment 
participants discussed the policy and programing lines for GEF-7. The eval- 
uation was the key document to inform these discussions. Following the 
meeting in which the participants scrutinized the final draft and asked some 
further questions for elaboration, the IEO finalized the OPS6 report, includ- 
ing a note by the independent advisory panel. It was then presented to the 
GEF Council meeting in December 2017 for endorsement. 

At that point, the evaluators’ job was done. IEO management participated 
in the third meeting in Brazil in January 2018 to be available for any clarifi- 
cations and answer any detailed questions. At this meeting, the participants 
thrashed out the Policy Agenda and Programming Directions documents 
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and the detailed allocation of funds between the different programs and focal 
areas for GEF-7. OPS6 fulfilled its purpose: throughout the meeting, the 
donors referred repeatedly to the document in making decisions between 
various alternatives. 

At the fourth and final meeting in Stockholm, all of the replenishment 
participants confirmed their pledges to the GEF trust fund for GEF-7. 


The OPS6 component evaluations and synthesis 


The evaluations underpinning OPS6 commenced after the replenishment 
in 2014 with the approval of the IEO work program and budget for the 
sixth replenishment. However, early in the GEF-6 replenishment period, the 
theme for OPS6 was not finalized and the IEO work program was designed 
more as a collection of interesting evaluation topics intended to address gaps 
in previous years or continue evaluations that had begun during the pre- 
vious replenishment period. A change in IEO management in fall of 2014 
initiated the actual planning for OPS6: the report was to be delivered in 
October 2017 with early findings due in March 2017. We had two years to 
make sure we had enough evidence on performance and impacts of the GEF 
and on the institutional issues addressing GEF governance and policies. We 
revisited the work program in fall 2015 and embarked on several studies 
simultaneously to be able to meet the timeline. A few work planning retreats 
helped us develop a theme for OPS6 that focused on the GEF’s role in the 
environmental finance landscape, review how various ongoing evaluations 
would contribute to the comprehensive evaluation, and assess the evaluation 
gaps we needed to fill. As described above, the approach paper* for OPS6 was 
developed through a broad consultative process and was approved at the June 
2016 GEF Council meeting. 

In response to the consultative process, OPS6 focused on two related 
themes: 


1 Relevance of the GEF strategy and the global contribution of the GEF, 
including governance and institutional issues 
2 Performance and impact of the GEF 


One of the main reforms that the GEF introduced during GEF-6 consisted 
of a set of pilot programmatic approaches aimed at addressing the main 
global environmental challenges through an integrated approach. This new 
approach included programing of GEF funds to help recipient countries meet 
their commitments to more than one global convention or thematic area by 
addressing the underlying drivers of environmental degradation. The GEF-6 
Programming Directions set out a rationale for the Integrated Approach Pilot 
(IAP) programs to address discrete, time-bound, global environmental chal- 
lenges in line with the targets and goals of the multilateral environmental 
agreements that the GEF serves (GEF 2014). 
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Three IAPs launched during GEF-6 and introduced this new dimension 
of programing that emphasized integration as a key organizing principle for 
GEF financing. These programs are structured around major drivers of global 
environmental degradation. Two programs are global, with one focused on 
urbanization (the Sustainable Cities IAP) and one on commodity-driven 
deforestation (the Commodities IAP); the third program addresses sustain- 
ability and resilience for food security in Sub-Saharan Africa drylands (the 
Food Security IAP). GEF financing for these programs was not “siloed” by 
focal area but was designed to be invested coherently to promote synergies 
in generating multiple global environmental benefits while also ensuring 
that any environmental progress does not negatively affect other related 
socioeconomic objectives. Leading into the seventh replenishment, the 
GEF Council was very eager for the IEO to provide early insights into the 
design and implementation of the IAPs, despite the limited time since their 
implementation. 

The IEO responded to the Council’s request for early insights into the 
design and implementation of the IAP programs. Because the IAPs were 
designed to enhance the GEF’s relevance and contribution to increasing 
global environmental benefits, the IAP evaluation addressed one overar- 
ching themes of OPS6: the GEF’s relevance and contribution. Since these 
programs were recently approved, we could not assess the performance, but 
design elements were reviewed that provided insights into the likelihood for 
good performance and sustainability. The next section presents the questions 
addressed under the two main themes addressed in OPS6: GEF relevance 
and contribution and the GEF’s performance and impact — and the evaluative 
sources of evidence. 


Theme 1: GEF relevance and contribution 


The themes addressed under the rubric of the relevance and the global con- 
tribution of the GEF included: 


e Global relevance and the relevance of the GEF-6 strategy with its focus 
on integrated programing 

e Focal area strategies, relevance to conventions, and results 

e Governance and health of the partnership 

e Development and implementation of cross-cutting policies on gender, 
safeguards, and indigenous peoples 

e Engagement with the private sector 

e The GEP’s attention to civil society organizations 

e Resource allocation mechanism (STAR) 

e GEF systems for results-based management and knowledge management 


Table 3.1 shows various questions addressed under the first theme and the 
sources of evidence used to address the questions. 


Table 3.1 Relevance and the global contribution of the GEF 


Key issue 


Evaluation questions 


Sources of evidence 


Global relevance of the 
GEF 


GEF-6 strategy 


Continuing relevance and 
effectiveness of the 
current GEF business 
model and health of the 
partnership 

Funding structure of the 
GEF 


Governance of the GEF 


Focal area strategies and 
results 


Multiple benefits of GEF 
support 


To what extent is the GEF relevant globally and how 
could its global relevance be enhanced? What would be 
the comparative advantage of the GEF in the changing 
landscape? 

To what extent is the GEF-6 strategy achieving its 
objectives? What does the early evidence suggest on 
the integrated approaches? 


To what extent is the current GEF business model 
effective and still relevant? Does the current business 
model optimize the capabilities within the GEF 
partnership? 


To what extent has the disproportionate share of 
funding flowing to climate change in recent years been 
addressed to create balance in GEF allocations to the 
focal areas? Have the issues related to the substantial 
donor areas been addressed? What are the implications 
of the interlinkages between the multilateral 
environmental agreements (MEAs) for GEF financing? 
How has the STAR allocation mechanism deepened 
country ownership? 

To what extent does the governance of the GEF 
continue to follow good practices? 

Are the focal area theories of change realistic? Do they 
align meaningfully with the objective of supporting 
integrated solutions? 

To what extent has the GEF support generated multiple 
benefits? 


Environmental/scientific literature; patterns of government 
spending; interviews with governments, international 
development agencies, and research institutions 


GEF-6 programing directions, evaluations of focal area 
strategies, country portfolio evaluations, formative 
evaluation of IAPs, evaluation of transformational change 
and multiple benefits 

Strategy documents, interviews, evaluation of multifocal 
area projects, a study on governance and relationships 
within the GEF 


Evaluation of the funding structure and resource allocation 


An evaluation on governance; the formative evaluation of 
the IAPs 

Focal area strategy evaluations, evaluations of the GEF’s 
results framework 


Evaluation of multiple benefits of GEF support 
(Continued) 
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Table 3.1 (Continued) 


Key issue 


Evaluation questions 


Sources of evidence 


Programmatic approaches 


Results-based 


management 


Gender mainstreaming 


Role of the private sector 


Role of civil society 
organizations and 
indigenous peoples’ 
participation 

Knowledge management 


What has been GEF’ experience with programmatic 
approaches? How effective have these approaches been 
in different contexts and what have they contributed to 
global environmental benefits? What is the early 
evidence on the IAPs? 

To what extent is the results-based management system 
in a position to capture the impacts of GEF 
interventions? To what extent does this system support 
adaptive management? 

To what extent have gender issues and evaluation of 
GEF gender policy effectiveness been mainstreamed 
into the GEF’s work since the development of the 
gender policy? 

To what extent has the GEF played a catalytic role in 
mobilizing private-sector financing in addressing global 
environmental benefits? How has the GEF engaged the 
private sector to identify opportunities and leverage 
them effectively? How is the non-grant instrument 
performing? 

What has been the role of civil society organizations in 
the GEF’s work? To what extent has the use of 
traditional knowledge been promoted in and by GEF 
activities? 

Is the GEF performing as a major data and information 
provider and are there any systemic issues to be 
addressed? What is the extent to which knowledge has 
been effectively managed and shared across the 
partnership? 


Evaluation of programmatic approaches, IAPs 


An evaluation of the GEF'’s results framework, results-based 


management system, and tracking tools 


Meta-analysis sub-studies on gender for OPS5, gender 
policy of the GEE project and country program 
evaluations 


An evaluation of the GEF’s engagement with the private 
sector at the project level and in the IAPs 


An evaluation of the involvement of civil society 
organizations at the project level 


A review of the GEF knowledge management strategy and 


the effectiveness of the implementation of the strategy 


og 
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Theme 2: program performance and impact 


The evaluations of program performance and outcomes/impacts considered 
the following key themes: 


e Outcome and sustainability ratings for completed GEF projects as emerg- 
ing from the 2013-2017 period 

e Progress toward impact of completed GEF projects for the period 
2013-2017 

¢ Trends in the catalytic role of the GEF as characterized by projects that 
focus on demonstration, scaling up, or investment 

e Trends in country ownership and the GEF’s role in contributing to pol- 
icy and regulatory improvements in countries 

e An understanding of the long-term impact of the GEF 

e Trends in performance issues including quality at entry, cofinancing, and 
supervision 

¢ Trends in implementation and achievement of the focal areas of the GEF 


During OPS5, a major effort was the assembly, clean-up, and validation of 
a database of GEF interventions. This database, with updates, served as a 
starting point for necessary OPS6 data on performance and progress toward 
impact. Thematic evaluations included in-depth analysis of GEF’s strategies 
and performance in the focal areas of biodiversity, climate change, chemicals 
and waste, international waters, and land degradation, as well as an analysis of 
programs that cut across focal areas with the potential for delivering greater 
environmental benefits through synergies. These evaluations and studies of 
GEF support to policy reforms and GEF support for transformational change 
provided evidence on the GEF’s long-term impacts. We conducted an ini- 
tial, formative evaluation of the IAPs to assess the relevance and coherence 
of their link to GEF-6 focal area strategies, their alignment with convention 
guidance, and the synergies between focal area strategies and country needs 
and ownership (GEF IEO 2018). This formative evaluation also looked at 
the IAPs’ initial uptake in participating countries and the efficiency of the 
launch process. During the GEF-6 period, the IEO also tested new evalua- 
tive approaches to measure the efficiency of GEF interventions in land deg- 
radation and biodiversity through a value-for-money analysis (see Chapter 4 
for a discussion of this approach). 

Table 3.2 presents the matrix of issues related to program performance and 
impact, with key evaluation questions and the sources of evaluative evidence. 


Bringing it all together into OPS6 


OPS6, the comprehensive report, addressed the role and comparative advan- 
tage of the GEF in the global landscape for environmental finance. The report 
discussed the issues of the GEF’s relevance to the conventions and countries; 


Table 3.2 Program performance and impact 


Key issue 


Evaluation questions 


Sources of evidence 


Continuing relevance of the GEF to 
multilateral environmental agreements 
(MEA) 


Project-level accomplishments 


Progress toward impact of completed GEF 
projects 


Catalytic role of the GEF 


Focal area achievements 


How relevant is the GEF to the guidance of 
the conventions, as emerging from the 
evaluations in the period 2013—2017? What 
are the implications of the focus on 
integrated approaches? 

What are the outcome and performance 
ratings on outcomes and sustainability of 
completed GEF projects for which 
terminal evaluations are available (2013- 
2017)? To what extent have the ratings 
improved? 

To what extent are the ratings on progress 
toward impact of completed GEF projects for 
the period 2013-2017 better or worse than 
the full cohort of OPS4 and OPS5 
completed projects? What are some 
of the factors responsible for the observed 
trends? 

What trends are discernible on the catalytic 
role of the GEF as characterized by 
foundation, demonstration, and/or 
investment projects? 

What are current trends in the 
implementation and performance of focal 
area support of the GEF as synthesized from 
thematic, country portfolio, and impact 
evaluations? 


Terminal evaluations of projects, country 
portfolio evaluations, thematic and impact 
evaluations 


Terminal evaluations of projects, country 
portfolio evaluations, impact evaluations, annual 
performance reports (APRs) 


Terminal evaluations of projects with review of 
outcomes to impact 


Country program, thematic, and impact 
evaluations; demonstration, foundation, and 
investment portfolio analysis; APRs; evaluation 
of transformational change 

Focal area strategies, meta-analyses based on 
thematic and impact evaluations, terminal 
evaluations, value-for-money analysis of GEF 
interventions 


ce 
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Key issue 


Evaluation questions 


Sources of evidence 


Country ownership and the GEF’s 
contribution to changes in country policy 
and regulations 

Addressing drivers of global environmental 
change 


GEF performance 


What trends are evident from the country 
portfolio evaluations with respect to country 
ownership? 

To what extent and in what forms has GEF 
support addressed drivers of environmental 
degradation (including positive and negative 
drivers)? What is the role of the GEF in 
policy matters such as UNFCCC and the 
SDGs? 

To what extent has performance in the GEF 
improved, especially with respect to the 
following? 


e The GEF project cycle 

e Co-funding 

e Management costs and fees 

e Quality at entry 

¢ Supervision 

e LDCF-SCCF 

What are the challenges in addressing these? 


Project terminal evaluations, country portfolio 
evaluations, evaluation of GEF support to policy 
reforms, thematic evaluations, impact evaluations 

Impact evaluations, thematic evaluations, APRs 


Portfolio analysis; terminal evaluations; APRs; 
LDCF/SCCF annual evaluation reports; 
governance, transformational change, and health 
of the partnership studies 
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the comparative advantage, performance, impact, and transformational 
change of the GEF; and the effectiveness of the institutional framework, gov- 
ernance, and policies. The IEO conducted a total of 29 evaluations over the 
GEF-6 replenishment period that provided the evidence for these themes and 
the related questions presented in Tables 3.1 and 3.2. Some evaluations were 
predominantly desk-based knowledge products, such as the evaluation of GEF 
support for transformational change, and several involved field visits to gather 
beneficiary data. We used a mix of qualitative and quantitative methods, 
including geospatial analysis, and formative approaches in the evaluations; we 
discuss these in Chapter 4. In all, the IEO visited 43 countries to collect data 
on the ground, analyzed 1,184 project terminal evaluations and covered a GEF 
portfolio of 4,433 projects — the entire GEF portfolio at the time. 

OPS6 and the component evaluations had limitations with respect to data 
and timing that are not unique to IEO evaluations. The terminal evalua- 
tions we analyzed often belonged to projects that commenced in previous 
GEF replenishment periods but still offered useful lessons. The GEF’s project 
management information system (PMIS) had incomplete and often outdated 
information on the status of various projects, requiring investment of con- 
siderable time and effort in ensuring the accuracy of project and monitoring 
data, including baselines and outcome information. Very little information 
was available on the implementation of the recently launched IAPs, yet the 
Council was very eager for information on this new programing approach. 
To mitigate some of the limitations, we used a formative evaluation approach 
focusing on the quality of design and early implementation of the IAPs, and 
we applied geospatial approaches to fill in data gaps on baselines, outcome, 
and impact information on important environmental variables of interest. 

Pulling together a comprehensive effort of this magnitude and doing so 
with quality requires following systematic and consistent processes and using 
appropriate evaluation methods. Every component evaluation — and the over- 
all OPS6 — followed the guidelines of good practice in evaluation process: 
the development of a concept note or approach paper, discussions with key 
stakeholders and reference groups, a peer-review process, competitive hiring 
of expert consultants, linking the findings and conclusions to draft recom- 
mendations, and discussion of recommendations with the GEF Secretariat 
and agencies prior to the finalization of the evaluation and presentation to the 
Council. While recommendations of the individual evaluations addressed the 
operational level, we drafted OPS6 recommendations at the strategic level to 
influence the GEF’s programing directions more broadly. 


OPS6 strategic conclusions and recommendations: 
inputs for the replenishment process 


The overall conclusions of OPS6 encompass the GEF’s relevance in the global 
environment; the performance and outcomes of GEF initiatives and policies; 
and the organization’s financing, governance, and systems (GEF IEO 2017). 
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Relevance in the global environment 


With its broad focus and as a financial mechanism for important environmen- 
tal conventions, OPS6 confirmed the unique space occupied by the GEF in 
the global environmental financing architecture. Despite limited funding, the 
GEF is the only institution that addresses global environmental issues beyond 
climate change alone, and it is the only global financial mechanism for several 
of the focal areas it addresses — including international waters, land degrada- 
tion, and chemicals and waste. The GEF is the principal financial mechanism 
for the Convention on Biological Diversity, the United Nations Framework 
Convention on Climate Change, the Stockholm Convention on Persistent 
Organic Pollutants, the United Nations Convention to Combat Desertification, 
and the Minamata Convention on Mercury. The GEF also funds projects in 
international waters and sustainable forest management that support the imple- 
mentation of a number of global and regional multilateral environmental agree- 
ments. As the financial mechanism for the Convention on Biological Diversity, 
the GEF is seen as a significant and reliable resource for funding for biodiversity, 
which attracts relatively few other funds. It also implements multifocal pro- 
jects and programmatic approaches in recipient countries to help them meet 
commitments to more than one global convention or thematic area by tack- 
ling underlying drivers of environmental degradation. OPS6 illustrated how 
these projects and programs are designed to promote complementarities and 
synergies in seeking multiple environmental benefits, while avoiding trade-offs 
between competing objectives. The GEF focal area strategies are also responsive 
to other major international environmental and development initiatives such as 
the SDGs of the 2030 Agenda for Sustainable Development. 

The GEF distinguishes itself by its ability to work through 18 agencies 
in more than 140 recipient countries. Through the STAR mechanism, and 
through programs and set-asides, these countries can access GEF resources 
to address environmental issues of national priority. GEF focal area inter- 
ventions are strongly aligned with country priorities, GEF support has often 
been instrumental in setting national priorities in the environmental sector, 
and the GEF provides unique and critical support for countries in meet- 
ing their obligations under various conventions. The OPS6 conclusions also 
addressed the focus of GEF funding between the middle income countries 
and the LDCs and SIDS. 


Performance and impact 


The component evaluations that contributed to OPS6 demonstrated the 
strong track record of the GEF in delivering overall good project perfor- 
mance, while also illuminating the extent to which sustainability is a chal- 
lenge. We explore the performance and impact of GEF programs and projects 
in detail in Chapter 5. Outcome and impact analysis confirmed that GEF 
interventions have contributed to decreasing environmental stress — biophys- 
ical changes that reflect reduction of threats emanating from human actions. 
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The GEF has played an important catalytic role and supported transfor- 
mational change primarily through mainstreaming; scaling up and market 
change have had limited success. Our analysis showed that transformational 
change occurs where projects aspire to drive change; market barriers are 
addressed through sound policy, legal, and regulatory reforms; private-sector 
engagement is encouraged through targeted capacity building and financial 
incentives; and mechanisms are put in place for future financial sustainability 
through the market, government budgets, or both. 

Chapter 5 also discusses specific impacts and areas for improvement asso- 
ciated with the GEF policies on gender, safeguards, and indigenous peoples. 
The Policy on Gender Equality has advanced the GEF’s efforts to strengthen 
gender mainstreaming in the organization’s programing and operations in a 
systematic manner. GEF policies and guidance on safeguards and indigenous 
peoples have supported efforts in these areas, but gaps exist relative to good 
practice in partner agencies and in implementation. 


Financing, governance, and systems 


The strategic conclusions in OPS6 addressed how exchange rate volatility, 
fragmentation in donor funding, and impediments to scaling-up non-grant 
instruments have affected GEF financing and, as a result, programs and allo- 
cations to countries. The study also identified how operational restrictions, 
lack of awareness of the GEF, and conflicting understandings and needs 
affected engagement with the private sector. 

Overall, OPS6 reported that the GEF partnership is well governed. 
Concerns continue to exist on matters related to representation, efficiency, 
accountability, and transparency, and the final OPS6 report explores these 
areas through both quantitative and qualitative data. From a systems stand- 
point, the comprehensive evaluation found progress in the GEF’s PMIS, 
results-based management system, and knowledge management; we also 
identified the ways in which the availability and quality of information in 
these systems needs further improvement. 

In summary, OPS6 confirmed that the changing landscape for environ- 
mental finance presented an opportunity for the GEF to build on its compar- 
ative advantage and make strategic choices. The establishment of new funds 
such as the Climate Investment Funds, the Green Climate Fund, the Asian 
Infrastructure Investment Bank, and the New Development Bank present an 
opportunity for the GEF to expand its presence in focal areas beyond climate 
change. Sources of comparative advantage for the GEF include its mandate to 
serve the conventions; its strong record of performance over 26 years; and its 
ability to address interlinkages and synergies across focal areas, implement 
policy and regulatory reforms in countries to create an enabling environment 
that attracts investment, deliver innovative financing models and risk-sharing 
approaches, and support lower income countries and SIDS. 
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Strategic recommendations 


Based on the findings and conclusions, the strategic recommendations of OPS6 
included: 


Strengthening strategic positioning 


The GEF should more strongly position itself in the focal areas that have 
limited financing sources and few players with the GEF’s depth of knowledge 
and experience. Within climate change, the GEF needs to redefine its niche, 
emphasizing its work with the enabling environment and measures to sup- 
port market transformation. Other important emphases are innovative pro- 
jects and piloting and demonstrating technologies and financial approaches. 
The GEF should explore its potential to be an incubator for countries to test 
and refine approaches prior to seeking large-scale financing through other 
partners. 


Promoting transformational change 


In any focal area, the GEF must design for transformation, promote policy 
and regulatory reform, and build institutional capacity in recipient countries. 
A project’s potential for transformation should be assessed at the design stage 
using clear criteria. 


Continuing the focus on integration 


Intervention design for integration must follow a strong, cogent ration- 
ale based on demonstrated additionality, GEF experience, GEF compara- 
tive advantage, innovative contributions, environmental need, and national 
relevance. 


Improving financial management 


The GEF needs to expand the number and variety of donors from both 
Organisation for Economic Co-operation and Development countries and 
middle-income countries. To secure its financing, the GEF should imple- 
ment foreign exchange risk management. 


Engaging the private sector 


The GEF should engage the private sector in areas beyond finance, such as by 
facilitating certifications and research and addressing sourcing and production 
practices along the supply chain. Long-term regulatory and policy intervention 
by the GEF can help prime the pump to catalyze private-sector investment. 
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Promoting gender equality 


The GEF’s Policy on Gender Mainstreaming must align more closely with 
international gender mainstreaming practice standards, with a comprehen- 
sive results or accountability framework and progress tracking. 


Revising safeguard policies 


Review of the policy on safeguards and rules of engagement with indigenous 
peoples should assess gaps against good practices and be updated accordingly, 
with supporting implementation and monitoring. 


Strengthening operational governance 


Establishing ground rules for cooperation among agencies will support mul- 
tifocal area efforts and program expansion, with clear communication of 
criteria for program, project, country, and agency selection. Addressing the 
potential for conflicts of interest is imperative. 


Improving systems 


Improvements to systems for project management information, results, and 
knowledge will enable the GEF to demonstrate its results and enhance learn- 
ing. Systems must deliver timely and accurate project information, good 
quantitative data on performance with a focus on impacts, and information 
to support project design, implementation, and monitoring. 


All of the recommendations were endorsed by the replenishment group and 
were reflected in the GEF-7 programing directions. 


Notes 


1. Officially the United Nations Conference on Environment and Development, at 
the time the largest gathering of heads of state and government to address a global 
challenge. 

2. The text of the Convention states: “...the global nature of climate change calls 
for the widest possible cooperation by all countries and their participation in an 
effective and appropriate international response, in accordance with their common 
but differentiated responsibilities and respective capabilities and their social and 
economic conditions” (UN 1992). 

3. Initially the United States until Japan became the largest donor. The formula was 
later loosened. 

4. www.gefieo.org/sites/default/files/documents/council-documents/c-50-me-07. 


pdf. 
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4 Evaluation methods 


Why we evaluate 


Evaluation plays an important role in the Global Environment Facility (GEF) 
in enhancing the effectiveness of the organization’s interventions in the pur- 
suit of global environmental benefits. At its simplest level, our evaluation 
seeks to address the question “Is this GEF project or program making a dif- 
ference?” Within the GEF specifically, the goal of independent evaluation is 
twofold (see Appendix 1): 


1 To promote accountability for the achievement of GEF objectives 
through the assessment of results, effectiveness, processes, and perfor- 
mance of the partners involved in GEF-financed activities; GEF results 
are evaluated for their contribution to global environmental benefits. 

2 To promote learning, feedback, and knowledge sharing on results and les- 
sons learned among the GEF and its partners as a basis for decision-making 
on projects, programs, program management, policies, and strategies; and 
to improve performance. 


The GEF Independent Evaluation Office (IEO) seeks to maintain a balance 
between the accountability and learning objectives. The purposes of evalu- 
ation in the GEF include understanding why, how, and the extent to which 
intended and unintended results are accrued, and their impact on stakehold- 
ers. Evaluation, when effective, is used to improve the design and perfor- 
mance of a planned or ongoing project or program (a formative evaluation); 
to make an overall judgment about the effectiveness of a completed project or 
program; and to generate knowledge about what works and why. Evaluations 
in the GEF feed into management and decision-making processes regarding 
the development of policies and strategies, and into the programming, imple- 
mentation, and reporting of activities, projects, and programs. Thus, evalu- 
ation in the GEF contributes to institutional learning and evidence-based 
policy-making, accountability, development effectiveness, and organizational 
effectiveness. It informs the planning, programming, budgeting, implemen- 
tation, and reporting cycle. In a nutshell, it aims to improve the institutional 
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relevance and achievement of results, optimize the use of resources, and max- 
imize the impact of the GEF investment. 

The IEO continuously strives to push the frontiers of environmental eval- 
uation through application of recently developed evaluation approaches, test- 
ing a variety of methods and using big data. Recent publications in peer 
reviewed journals reflect IEO contributions to advancing the evaluation 
approaches in sustainability: 


e Exploring the Socioeconomic Co-benefits of Global Environment Facility 
Projects in Uganda Using a Quasi-Experimental Geospatial Interpolation 
(QGI) Approach, Journal of Sustainability (Runfola et al. 2020) 

e The Use of Remote Sensing Analysis for Evaluating the Impact of 
Development Projects in the Yellow Sea Large Marine Ecosystem, Journal 
of Sustainability (Sidman et al. 2020) 

e A Joint Evaluation with Lessons for the Sustainable Development Goals 
Era: The Joint GEF-UNDP Evaluation of the Small Grants Programme, 
American Journal of Evaluation (Carugi and Bryant 2019) 

e Improving International Development Evaluation through Geospatial 
Data and Analysis, International Journal of Geospatial and Environmental 
Research (Lech et al. 2018) 


How we evaluate 


Evaluations in the IEO cover a broad spectrum ranging from project-level 
validations to comprehensive evaluations that address a broad spectrum of 
institutional and impact themes and involve a variety of quantitative and 
qualitative methods (see Box 4.1). 

Our evaluation approaches in the IEO have evolved to adapt to changes 
in GEF interventions and in response to the Council’s needs for information 
on recently designed pilot programs and real-time information. For exam- 
ple, while the GEF continues its interventions in focal areas, it is shifting 
toward more multifocal projects and integrated programming to address the 
drivers of environmental degradation. With the advent of the Sustainable 
Development Goals (SDGs), recognition of the synergies (and/or trade-offs) 
between environmental and human systems has grown. To accommodate 
these shifts, the IEO has adjusted its work program to provide insights into 
the early design and implementation of the integrated programs and to delve 
into evidence on socioeconomic outcomes. This has involved expanding the 
suite of evaluation types and the methodologies we employ, as reflected in the 
GEF Evaluation Policy (see Appendix 1). 

We have used formative evaluation methods to assess the integrated 
approach pilots (IAPs) for relevance and design before their full roll-out. 
We conduct process/implementation evaluations to determine whether 
policies — such as those on gender, safeguards, and indigenous peoples — or 
programs have been implemented as they were intended. Our outcome/ 
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Box 4.1 


Evaluations conducted by the IEO 


Validations of project evaluations at the completion of the project 
implementation (terminal evaluation), mainly conducted by GEF 
Agencies. 

Program evaluations of a coherent set of interventions to attain specific 
global, regional, country, or sector objectives; these include evaluations 
of the GEF focal areas, programmatic approaches, and GEF corporate 
programs. 

Performance evaluations of the GEF’s portfolio of completed projects 
to assess the relevance, efficiency, and effectiveness in delivering the 
expected results. 

Country and country cluster evaluations of one or more agencies’ 
portfolio of projects and activities, and the assistance strategy behind 
them, in a country or cluster of countries. Strategic cluster evaluations 
focus on a limited set of strategic issues across country portfolios. 

Process evaluations of the functioning of participating organizations, 
instruments, mechanisms, and management practices; these include 
evaluations of institutional and procedural issues across GEF focal areas 
and assessments of experience with GEF policies, criteria, and pro- 
cedures. Process evaluations may be conducted during the design or 
implementation of an intervention. 

Impact evaluations of the long-term effects produced by an intervention, 
intended or unintended, direct or indirect. Impact may be assessed at 
the project, program, portfolio, ecosystem, or country level and includes 
global environmental benefits. 

Thematic evaluations of a selection of interventions addressing a specific 
theme, issue, or focal area across the GEF portfolio; these include evalu- 
ations that assess cross-cutting issues. 

Comprehensive evaluations of the GEF (commonly referred to as an 
overall performance study, or OPS) inform the GEF replenishment 
cycles and address performance and results in terms of global environ- 
mental benefits of GEF projects and programs. They also assess GEF 
institutional arrangements, policies, strategies, and priorities. The ear- 


lier evaluations feed into the comprehensive evaluations. 


effectiveness evaluations measure project and program effects by assessing 
the progress in environmental outcomes against the approved objectives. 
Finally, our impact evaluations assess the long-term net effectiveness of the 
GEF’s programs in achieving their ultimate goals of delivering global envi- 
ronmental benefits. The units of analysis for these various evaluations are 
projects, programs, or countries. Or, an evaluation could address a specific 
theme, issue, or focal area that cuts across the entire GEF portfolio. We bring 
together all of these individual evaluations in preparing the comprehensive 
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Box 4.2 


OECD DAC criteria 


Relevance: Is the intervention doing the right things? 

The extent to which the intervention objectives and design respond to ben- 
eficiaries’, global, country, and partner/institution needs, policies, and 
priorities, and continue to do so if circumstances change. 

Coherence: How well does the intervention fit? 

The compatibility of the intervention with other interventions in a coun- 
try, sector, or institution. 

Effectiveness: Is the intervention achieving its objectives? 

The extent to which the intervention achieved, or is expected to achieve, 
its objectives; the intervention’s results, including any differential results 
across groups. 

Efficiency: How well are resources being used? 

The extent to which the intervention delivers, or is likely to deliver, results 
in an economical and timely way. 

Impact: What difference does the intervention make? 

The extent to which the intervention has generated or is expected to gen- 
erate significant positive or negative, intended or unintended, higher 
level effects. 


evaluation of the GEF, conducted every four years as part of the funding 
replenishment cycle, to provide strategic insights into the performance and 
results of GEF projects and programs, and into GEF institutional arrange- 
ments, policies, and priorities. 

Various evaluations address a broad spectrum of questions. These may per- 
tain to identifying where a problem lies; a problem’s definition, scope, and 
scale; the appropriateness of an intervention to address a problem; the GEF’s 
additionality; or a project’s or program’s outcome effectiveness, impacts, 
and potential for transformational change. To provide a systematic lens to 
address such questions, we, like other global development institutions, apply 
the framework of evaluation criteria from the Organisation for Economic 
Co-operation and Development’s Development Assistance Committee 
(OECD DAC; see Box 4.2). The purpose of the evaluation criteria is deter- 
mining an intervention’s merit, worth, or significance. These criteria aid us 
in looking at the processes by which change happens and at the results of the 
change (OECD DAC 2019). 


How we evaluate: framework and recent innovations 


We conduct a broad spectrum of evaluations in the IEO as presented in 
Box 4.1. Within the context of the GEF, project- and program-level ter- 
minal evaluations are the responsibility of the implementing agencies and 
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are mandated according to the GEF evaluation policy and the minimum 
standard requirements. The data from the terminal evaluations underpin all 
performance, thematic, country, and impact evaluations the IEO conducts. 
Field visits, interviews with stakeholders, geospatial analysis, and surveys are 
other important sources of evidence in the thematic evaluations we conduct. 
We continue to develop and apply new approaches and tools as GEF pro- 
gramming evolves and as the field of evaluation progresses. We have recently 
developed frameworks for assessing project and program additionality, trans- 
formative change, and efficiency — core themes at the heart of the GEF 2020 
and GEF-7 strategies — and have mainstreamed the use of geospatial analysis 
within our suite of evaluation tools. 


Terminal evaluations 


The implementing agencies conduct terminal evaluations according to the 
relevant GEF Guidelines (GEF IEO 2017). The guidelines cover the roles and 
responsibilities for the agencies, evaluation units of the agencies, the IEO, 
and operational focal points, and address the content of the evaluation report. 
The terminal evaluations are required to assess: 


e project theory of change (TOC) 

e effectiveness along the OECD DAC criteria 

e progress toward impact 

e sustainability 

e design and implementation of monitoring and evaluation 
e quality of implementation and execution 

e lessons learned 


The terminal evaluation guidelines also provide the criteria for rating vari- 
ous performance dimensions. Given the importance of these evaluations, the 
IEO validates the terminal evaluations of GEF projects from all agencies.! We 
aggregate the ratings from the terminal evaluations along various dimensions 
and report them in the Annual Performance Report of the IEO — the aggre- 
gate report card for the GEF. With the shift toward programs, the terminal 
evaluations for the “child projects” (individual projects under the umbrella 
of a larger program) must follow the terminal evaluation guidelines, which 
require the child projects to demonstrate contribution toward the larger 
program. At the program level, the terminal evaluations should demonstrate 
both overall effectiveness and generation of program additionalities beyond 
the sum of the contributions of the individual projects. 


Additionality: what environmental difference did our funds make? 


A central concern for the GEF, as for other development institutions, is 
the attribution of its support to environmental impact. Whether funding 
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programs directly or through other implementing agencies, most develop- 
ment institutions focus on increasing the total flow of resources toward a 
particular cause. The GEF IEO must evaluate the additionality that GEF 
funding generates. In other words, did GEF funds displace other funding that 
would have materialized? Equally important, what outcomes can truly be 
attributed to the GEF funding, and what part of these outcomes would have 
happened even without additional funding? 

The GEF has for several years addressed these considerations through 
the incremental cost approach. The aim is to ensure that GEF funds do not 
substitute for existing development financing, instead providing additional 
funding to produce agreed-upon global environmental benefits. For many 
institutions, the GEF still serves as the leader in defining additionality and 
pointing to an implementable framework. Institutions also recognize that the 
GEFF is a special case in which the baseline (i.e., the without-GEF scenario) is 
expected to show additionality compared with the current situation in which 
no intervention has taken place. Thus, while typical projects assess their 
impact based on a counterfactual and a baseline analysis, the GEF defines 
its additionality as the incremental contribution from the GEF involvement 
above and beyond the additionality that would have occurred in the absence 
of the GEF. This “double increment” has most often been used in the climate 
finance area related to the offsets of emissions of greenhouse gases and to the 
program of payment for ecosystem services. 

However, our 2006 evaluation of the incremental cost approach (GEF IEO 
2006) found that it added little to the operational aspects of project prepara- 
tion, was often poorly understood in its concept, and, at times, could even 
lead to operational modifications that ran counter to other environmental 
benefits or good development practices. The GEF is a unique partnership 
that thrives on pursuing environmentally effective projects with attribut- 
able outcomes, while also bringing to bear its influence and the agreements 
reached through global environmental conventions in shaping how global 
environmental commons are protected and nurtured. Therefore, the GEF’s 
environmental additionality cannot be measured with a one-size-fits-all 
approach but requires recognition of the direct and indirect ways the GEF 
shapes the impact on global environmental benefits through its financial and 
nonfinancial roles. 

In parallel, the literature and practices in assessing additionality have 
evolved significantly toward accounting for factors beyond the immediate 
project objectives. Considering that the incremental contribution arising 
from the work of development institutions (multilateral, bilateral, and civil 
society and nongovernmental organizations) often remains difficult to define, 
the existence of a significant body of literature and a variety of practices to 
draw on is not surprising. The term “additionality” is based on the project 
and program evaluation principles of establishing a strong counterfactual to 
derive the true impact on development outcomes of a project or program. At 
the project level, extensive literature has been developed in recent years with 
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the help of multilateral development banks.” Establishing the preconditions 
to accurately assess the impact of projects requires early development of mon- 
itoring systems and clear counterfactuals — a challenge for most development 
institutions. Beyond the general agreement on the relation to a counterfac- 
tual, however, little progress has been made in reaching a common defini- 
tion, or measurement (Oxfam International 2017). 

Institutions are also recognizing the value of separating financial addition- 
ality — such as drawing private-sector investment into solving developmen- 
tal problems through commensurate public policies or investments — from 
developmental additionality, such as regulator reform, capacity building, 
and other factors associated with positive long-term development outcomes. 
The OECD DAC, for example, studied carefully the role of drawing in pri- 
vate-sector investments through public development interventions (Benn 
et al. 2017). 

In light of the need for a more robust approach to assessing the GEF’s envi- 
ronmental and other additionalities, and the recent evolution toward a more 
encompassing definition of additionality, the IEO developed a framework 
to align the additionality concept with current strategies and practices (GEF 
IEO 2018a). 

Additionality is defined in this framework as: 


e Changes in the attainment of direct project outcomes at project com- 
pletion that can be attributed to GEF interventions; these can be 
reflected in an acceleration of the adoption of reforms, the enhancement 
of outcomes, or the reduction of risks and greater viability of project 
interventions. 

e Spillover effects beyond project outcomes that may result from systemic 
reforms, capacity development, and socioeconomic changes. 

e Clearly articulated pathways to broaden the impact beyond project com- 
pletion that can be associated with GEF interventions. 


The six dimensions of the GEF’s additionality are presented in Box 4.3. We 
now apply these regularly in the IEO’s thematic evaluations, and the GEF 
encourages application of the framework at the project design stage. The 
evaluation policy now requires clear articulation of the GEF’s additionality. 
A study demonstrating the utility of the additionality framework is an 
evaluation of the GEF-United Nations Industrial Development Organization 
(GEF-UNIDO) Global Cleantech Innovation Programme (GCIP), released 
in April 2020 (GEF IEO 2020). Serving small and medium enterprises, this 
program encourages cleantech entrepreneurial solutions to pressing environ- 
mental and economic challenges. To provide stakeholders with insights into 
the GCIP and lessons for similar future projects and programs, the evaluation 
assessed the program’s relevance, additionality, outcomes, and sustainability 
in eight countries. The evaluation framework supported a mixed-methods 
approach that drew on internal and external factors, including document 
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Box 4.3 


Six areas of the GEF’s additionality 


Specific environmental additionality: The GEF provides a wider range 
of value-added interventions and services to achieve the global environ- 
mental benefit. 

Question: Has the project generated global environmental benefits that 
would not have happened without the GEF’s intervention? 

Legal/regulatory additionality: The GEF helps stakeholders achieve 
transformational change to environmentally sustainable legal/regulatory 
reforms. 

Question: Has the project led to legal or regulatory reforms that would not 
have occurred in the absence of the project? 

Institutional additionality/governance additionality: The GEF sup- 
ports the transformation of existing institutions to increase their effi- 
ciency and impact on environmental sustainability. 

Question: Has the project strengthened institutions to provide a support- 
ive environment for achievement and measurement of environmental 
impact? 

Financial additionality: The GEF provides incremental funding to trans- 
form projects’ benefits from a national/local scale to a global scale. 

Question: Has the involvement of the GEF led to greater flows of financing 
than would otherwise have been the case from private-or public-sector 
sources? 

Socioeconomic additionality: GEF activities help individuals improve 
their livelihood and social benefits. 

Question: Can improvements in living standards among population 
groups affected by environmental conditions be attributed to the GEF 
contribution? 

Innovation additionality: The GEF provides efficient/sustainable tech- 
nology and knowledge to overcome existing social norms/barriers/ 
practices. 

Question: Has the GEF involvement led to a fast adoption of new technolo- 
gies, or the demonstration of market-readiness for technologies that had 
not previously demonstrated their market viability? 


review and interviews with GEF Secretariat and UNIDO staff, government 
representatives, and private-sector stakeholders. 

The evaluation noted the GEF’s additionality in several areas through its pro- 
motion and results in innovation for clean technology, socioeconomic returns, 
institutional capacity, realization of financing for some startups, and business 
support to enterprises, the products and services of which have environmental 
benefits. However, GCIP did not realize policy and regulatory strengthening 
additionality in a meaningful way because these project activities were limited, 
under-resourced, and generally begun at a later stage of implementation. 
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GCIP’s focus on cleantech business acceleration was new, unique, and a 
value-add for the innovation ecosystem. Despite recognition that the clean- 
tech concept could stimulate economic growth, few such projects existed, and 
those that did tended to operate in silos. GCIP business assistance (primarily to 
early-stage entrepreneurs) filled a gap. One-to-one interactions with mentors 
under the GCIP allowed for tailored advice and were conducive to the develop- 
ment of close, long-term relationships between mentors and teams. The strat- 
egy of approaching alumni (entrepreneurs who had “graduated” from previous 
GCIP rounds) to serve as mentors, judges, and local trainers was novel, and 
many who took up these roles were motivated to give back to other early-stage 
entrepreneurs. The start-ups had privileged access to local private experts. GCIP 
stressed the importance of a risk-taking mindset and encouraged local solutions. 
Box 4.4 shows an example of a GCIP participant enterprise. 


Box 4.4 


Pakistani solution for low-cost, quickly built housing 


Karachi-based ModulusTech designed mobile, low-cost, earthquake- and cyclone- 
resistant, energy-efficient housing that can be assembled within a few hours, with 
plumbing and electricity included. Having set its sights on solving housing prob- 
lems for displaced people, the team originally thought about targeting interna- 
tional charities and the refuge community. To its surprise, the company found its 
first customer in the mining industry, where its innovation is being used to fulfill 
a need for temporary accommodation. 
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GCIP also promoted indigenous technology development, which allowed 
for reduced costs of equipment for the entrepreneurs and easier adoption for 
urgent development priorities such as access to clean and affordable energy, 
clean water, and climate resilient agriculture. The program fostered social 
inclusiveness in ways such as including youth and Black entrepreneurs in 
South Africa, and women in Pakistan. Under the GCIP framework, entre- 
preneurs have developed and commercialized ideas with meaningful social 
and economic impact. Examples include innovations in sanitary pads (devel- 
oped by Gracious Nubian in South Africa and Saathi Eco Innovations in 
India) that have provided affordable solutions for women and girls, especially 
in rural areas, who were previously prevented from working and attending 
school during menstruation. GCIP was able to leverage private-sector finance 
to support promising cleantech solutions. The start-ups had more success in 
gaining access to venture capital through the national-level investor. 

A second evaluation in which we applied the additionality framework was 
the Evaluation of GEF Support to Mainstreaming Biodiversity, completed in 
December 2019 (GEF IEO 2019a). The evaluation was based on a portfolio 
analysis of 471 relevant projects and in-depth case studies from Colombia, 
India, and South Africa, and used methods including review of documenta- 
tion and literature; site visits; and interviews with key stakeholders such as 
government officials, implementing and executing agency staff, civil society 
organizations, and project beneficiaries. The evaluation found that the GEF 
biodiversity mainstreaming portfolio has contributed to legal, regulatory, 
governance, and socioeconomic additionalities that go beyond incremental 
cost benefits. These include innovative approaches based on multistakeholder 
partnerships linking grassroots organizations to regional research institu- 
tions, advocacy platforms, and national environmental authorities. Landscape 
management practices are validated on the ground and elevated to influence 
national policy and legislative-regulatory reform. Several projects have con- 
tributed to landmark biodiversity legislation; transformed core institutional/ 
sector practices; and achieved measurable conservation impacts in forest 
cover, pasture, and other biodiversity indicators. However, capturing other 
additionalities — such as socioeconomic and environmental impacts deriving 
from the GEF’s support for biodiversity mainstreaming in productive land- 
scapes and seascapes — has been a challenge. 


Transformational change: moving 
beyond the incremental 


As the GEF’s vision statement (GEF 2012) lays out, the unprecedented nature 
of the pressures faced by the earth’s ecosystems in the coming decade means 
that incremental environmental strategies alone will not suffice. These pres- 
sures “compel the GEF to equip itself to promote transformational change.” 
The GEF 2020 Strategy Paper identifies market transformation as one of the 
areas requiring a systematic effort to capture lessons learned from past project 


50 Evaluation methods 


experience and to leverage this knowledge to guide users and scale up the 
GEF’s impact (GEF 2015). 

To better understand what drives transformational change and to improve 
the identification, design, and implementation of future operations to sup- 
port transformational change, the IEO developed a framework (see Chapter 
6, Figure 6.2). The transformational change mechanism generates catalytic 
effects through mainstreaming, demonstration, and replication and can 
result in changes at the market or systems level. Internal factors influencing 
transformational change are the quality of implementation and execution, 
capacity building, and donor partnerships. External factors driving change 
include government ownership and support, implementation capacity of 
local institutions, adequacy of the policy environment, civil society, pri- 
vate sector and local community participation, and enabling economic and 
market conditions. We have applied this framework in several evaluations, 
including the previously discussed biodiversity and the cleantech evaluations, 
to understand whether and how GEF projects have been transformative. 


Application of the transformational change framework 


Lighting Africa: promoting market-based solutions to advance energy access 


About 600 million people in Africa have no access to grid electricity, and 
this number is expected to rise to about 700 million by 2030. These people 
rely on polluting and dangerous sources of lighting such as kerosene lamps, 
candles, and battery-powered torches. Fuel-based lighting is generally of low 
quality and expensive, impeding learning and economic productivity. 

Given advances in technology and increased competition, portable modern 
lighting devices have become more affordable. This created an opportunity 
for people living in off-grid areas to replace kerosene lamps with higher qual- 
ity, safer, and more affordable modern lighting products such as solar lamps. 
However, despite the benefits of solar lamps, the market was not develop- 
ing as quickly as expected. To understand why, the International Finance 
Corporation (IFC)/World Bank undertook a market appraisal (World Bank 
Group 2007) and identified six barriers inhibiting market growth: 


e Consumers did not trust the solar products available. Some solar lamps 
were already available in the market when the Lighting Africa program 
began, but many of these products were poorly made and did not work 
properly. 

e Consumers did not know the benefits of solar lamps, how to use them, 
or where to buy them. Some consumers were unaware that solar lamps 
existed. 

e Manufacturers and designers did not know consumer preferences for the 
design and function of a solar lamp. 

e Supply chain entities did not know each other. Solar lamp manufactur- 
ers entering the market to serve lower income consumers in developing 
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countries did not have an established distribution network and were 
unsure how to identify reliable distributors. 

e Lack of finance was a big problem. Designers and manufacturers, distrib- 
utors and importers, and retailers needed financing to purchase and move 
products to the end users. Lower income consumers needed microloans 
to help with the upfront cost of purchasing a solar lamp. 

e Long customs processes and import tariffs on solar lamps were common 
concerns among manufacturers who considered importing solar lamps to 
African markets. 


The Lighting Africa program was created to transform the off-grid market 
by removing these barriers. Its goal was to help catalyze markets for quality, 
affordable, clean, and safe off-grid lighting and, ultimately, to create a sustaina- 
ble commercial platform that would realize the vision of providing 250 million 
people with modern off-grid lighting by 2030. The overall approach was 
to demonstrate the viability of the market by providing market intelli- 
gence, developing a quality assurance infrastructure, facilitating business- 
to-business interactions, helping governments address policy barriers, pro- 
viding business development services, and facilitating access to finance for 
manufacturers, local distributors, and consumers. The program received 
about $22 million in donor contributions from 2007 to 2013, with the GEF 
the largest donor, providing more than one-third of the funds (World Bank 
and Independent Evaluation Group 2015).* 

In 2014, the final evaluation of the Lighting Africa program concluded that 
the program had played a crucial role in transforming the market (Castalia 
Strategic Advisors 2014). The program was effective and made an impact. A 
few of the key accomplishments were: 


e Through the program’s quality assurance efforts, 183 solar lamp models 
were tested and 66 received the Lighting Africa quality certification. 

e The program hosted 1,157 forums during its consumer education cam- 
paigns, directly reaching more than 36,000 people in Kenya. 

e More than 680,000 Lighting Africa-certified lamps were sold in Kenya, 
135 percent more than the Kenya program’s target. Furthermore, almost 
2 million lamps were reported to have been sold in other African coun- 
tries — 185 percent above the target. More work is needed to deter- 
mine the extent to which these sales can be attributed to the program, as 
noted in the evaluation, because this estimate does not take into account 
a counterfactual of what would have occurred without the program. 
However, interviews with retailers, consumers, and manufacturers con- 
firmed that Lighting Africa was a very important influence on market 
development, so the true impact may well have been higher. 


The evaluation also concluded that the benefits achieved by the program 
were sustainable after donor funding stopped, with interviews suggesting 
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that people who have used solar lamps will continue to do so and suppli- 
ers will continue to supply. The extent to which the market transformation 
process itself will continue, however, remains to be seen. Although the pro- 
gram has laid the groundwork for continued market transformation through 
arrangements with an industry association and a Kenyan nongovernmental 
organization to take over and continue the program activities, these organi- 
zations are still partially reliant on donor support. 

Based on the findings of the evaluation, three main factors were instru- 
mental in Lighting Africa’s success, and their maintenance will be essential 
for continuing the transformation: 


e The first and obvious success factor was the program’s operation in areas 
with a proven, strong demand for improved off-grid lighting solutions. 

e The second was having a carefully designed set of interventions that 
simultaneously targeted all major market barriers. Because barriers differ 
from market to market, the program started with a basic program design 
but tailored the components to address the specific barriers identified in 
the target countries. 

e The third was the program’s focus on market transformation. The 
Lighting Africa program did not fund solar lamps — it funded activi- 
ties that created effective markets in which consumers spent their own 
money to buy solar lamps. To sustain this success factor, organizations 
must resist the temptation to purchase lamps for poor people and instead 
vigorously pursue pro-market interventions such as microfinance to assist 
the purchase of solar lamps. 


Choosing our tools: evaluation methods 


The evaluation questions we pursue guide our choice of evaluation methods. 
Most IEO evaluations aim to either determine project effectiveness after imple- 
mentation or improve project design and implementation as they are develop- 
ing. Are GEF interventions relevant? Do GEF interventions work or not work? 
Why do they work? What are the factors influencing outcomes and perfor- 
mance? How do these factors influence each other within the system of inter- 
est? To answer these questions, we often apply both quantitative and qualitative 
methodological approaches while adopting a systems approach to evaluation. 


A systems approach to evaluation 


Our evaluations typically begin with developing a TOC to establish the 
link between GEF interventions and their intended outcomes. This linear 
approach, however, is rather simplistic. GEF projects operate in complex 
systems marked by interconnectedness among myriad factors. Within these 
systems, interventions may have unintended or secondary consequences that 
could affect impacts but may not be considered — or cannot be — during 
project design or within the linear TOC. Thus, IEO evaluations go beyond 
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the intervention’s TOC to examine the complex system within which 
the intervention has taken place. We also consider specific elements of an 
intervention’s TOC and the larger system in determining which evaluation 
questions need to be asked. This systematic basis allows us to use appropriate 
evaluation methods and adapt our approaches to integrate new or evolving 
knowledge throughout the process (Garcia and Zazueta 2015). 


Mixed methods 


Quantitative data provide rich information on the “what” and the “who” 
of a project or program — the outcomes, reach, and cost efficiency — and are 
collected through the GEF’s results monitoring system. This system incor- 
porates surveys conducted during an evaluation and, more recently, remote 
sensing and satellite data. Our suite of quantitative tools includes: 


e simple statistical measures to observe patterns of performance 

e multivariate regression analysis 

e regression discontinuity analysis 

e  quasi-experimental approaches to compare GEF units with those that 
have not received GEF assistance 

e machine learning algorithms and causal trees to explain factors influenc- 
ing environmental outcomes 

e valuations of carbon stocks 


Several IEO evaluations now apply remote sensing and geospatial methods, 
which we have found to be useful and innovative tools for measuring envi- 
ronmental impact. These tools provide reliable and cost-effective baseline 
information that is often missing in project documents, help detect changes 
over long periods of time, and assist in tracking progress toward the achieve- 
ment of environmental and other development targets (Lech et al. 2018). Our 
applications of geospatial data include: 


e better understanding the relevance of GEF interventions 

e measuring returns on GEF investments 

e assessing environmental effectiveness on forest cover, habitat quality, and 
carbon sequestration 

e evaluating socioeconomic co-benefits of GEF projects 

e addressing the long-term impacts of GEF interventions through sustain- 
ability analysis 


Figures 4.1 and 4.2 illustrate the application of geospatial data in biodiversity 
to answer important evaluation questions. Figure 4.2 presents how geospa- 
tial analysis is used to assess the relevance of GEF interventions. We cre- 
ated a spatial overlay of globally distributed GEF-supported protected areas 
with sites of conservation importance. This geospatial analysis shows that the 
GEF-supported protected areas are located in biodiversity hot spots (GEF 
IEO 2016). Figure 4.3 illustrates the impact of GEF projects by measuring 
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Figure 4.1 Explaining geospatial analysis 


forest cover change in protected areas. Over the last 25 years, the GEF has 
provided more than $3.4 billion in grants and an additional $12 billion in 
raised cofinancing for the protection of almost 2.8 million km? of the world’s 
terrestrial ecosystems. A substantial portion of the GEF’s support is dedicated 
to strengthening protected areas and protected area systems and ensuring 
their sustainability. Our impact evaluation of the GEF’s long-term support 
to protected areas and protected area systems using remotely sensed satellite 
data analysis demonstrated that GEF-supported protected areas had, in gen- 
eral, better conservation outcomes (less forest loss) compared to their buffers 
— protected areas that were not supported by the GEF — and compared to the 
overall respective country averages (GEF IEO 2016). 
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Figure 4.2 Use of geospatial analysis to assess relevance of GEF’s biodiversity interventions 
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Figure 4.3 Using geospatial analysis to measure impacts: analysis of forest cover change 


from geospatial data 
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Figure 4.3 uses an example from a protected area in Mexico to illustrate 
the forest change data used for the global analysis. It shows the protected area 
boundary and the buffer area; the forest cover for the base year; and charts of 
forest cover, forest gain, and forest loss. 

Together, applying geospatial data and our array of quantitative methods 
has helped us address common methodological challenges such as lack of 
baseline data, low survey response rates, sampling bias, difficulties in select- 
ing appropriate counterfactuals, and accounting for the impact of multiple 
scales and contexts on processes and interventions. This has strengthened our 
evidence, allowing us to deliver robust analysis. 


Application of geospatial analysis to measure 
efficiency: a value-for-money analysis 


For development interventions and projects, particularly in a situation of scarce 
resources, a key question among donors remains: do interventions deliver 
value for money? To address this question for GEF interventions, we brought 
together economists, computer scientists, and geographers with expertise in 
remote sensing and impact evaluation to apply a value-for-money (VFM) 
assessment to the case of GEF land degradation and biodiversity projects 
(GEF IEO 2018). This approach explicitly quantified the causally identified 
impact attributable to GEF project locations using three indicators: capturing 
vegetation productivity, forest fragmentation, and forest cover change. It also 
demonstrated the VFM achieved by these GEF projects in terms of carbon 
sequestration. The research team conducted a series of quasi-observational 
experiments contrasting land degradation and biodiversity project locations 
with geographic locations at which no known intervention occurred and 
that were similar in qualities such as initial environmental state, proximity to 
infrastructure, and environmental characteristics. We used these contrasted 
locations in conjunction with hybrid econometric propensity score matching 
and machine-learning techniques to account for both potential variation in 
treatment effects across different sociopolitical and environmental conditions 
and uncertainty in underlying assumptions and data. 

Recent work (see Athey and Imbens 2015; Shen et al. 2016; The PLOS 
ONE Staff 2014) has illustrated that, with certain adjustments, evaluators 
can use machine learning approaches to identify how the causal effects of 
an intervention (e.g., international aid, a medical treatment) vary across key 
parameters. This approach’s relevance in top-down or global-scope analyses 
makes it applicable to GEF projects because they are unlikely to have the 
same effect across highly variable geographic contexts, and the drivers of 
such variation may not be known. We leveraged a wide range of environ- 
mental, socioeconomic, and project characteristic covariate information to 
ensure that comparisons were between similar sets. This included distance 
to roads, rivers, urban areas (travel time), nighttime light intensity, slope, 
elevation, temperature, and precipitation ranges. It also included geographic 
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factors such as latitude and longitude to promote matches that were reason- 
ably geographically proximate — matched locations were within 50—250 km 
of each other. After constructing impact estimates following the causal tree 
approach, we estimated valuations in a two-step procedure: 


1 First, we used the National Aeronautics and Space Administration’s 
(NASA’s) carbon storage data set and the Intergovernmental Panel on 
Climate Change’s (IPCC’s) Tier-1 Global Biomass Carbon Zones to 
translate the impact of GEF projects on the three indicators into esti- 
mates of carbon sequestration using a linear modeling approach that 
accounts for regional differences in the relationship between flora and 
the indicators. 

2 We then used value transfer to approximate valuations for both carbon 
sequestration and biodiversity. In this approach, the value of nonmar- 
ket services is approximated based on previous studies of similar services. 
Although primary data collection on valuation can provide strong, in-situ 
measurements of valuation, evidence suggests that the density of literature 
on similar services — and the cost-effective nature of the value transfer 
approach — positions value transfer as a strong second-best strategy. 


Land degradation impacts 


This study identified a global positive impact of GEF land degradation pro- 
jects along all indicators examined, but also noted considerable heterogeneity 
in these impacts across different geographic contexts. Findings included: 


e A lag time of 4.5-5.5 years was an important inflection point at which 
larger impacts were observed. 

e Projects with access to electricity tended to have some of the largest rel- 
ative positive impacts. 

e The initial state of the environment is a key driver in GEF impacts, with 
GEF projects tending to have a larger impact in areas with a poor initial 
condition. 

e Projects in Africa and Asia had generally positive impacts on average. 

e Projects in Latin America and the Caribbean, North and South America, 
and Oceania all had positive impacts on all three indicators. 


The analysis identified a range of values consistent with previous analyses 
of the value of land degradation projects. Because considerable uncertainty 
exists, the range of potential benefits from a single—focal area land degrada- 
tion project is estimated at $52—$143/ha affected in terms of carbon seques- 
tration alone; soil retention promotes an additional value of $10—$43/ha, for a 
total valuation of $62—$186/ha across degradation projects. After accounting 
for costs, the IEO estimated that the per-dollar return on investment for land 
degradation projects is approximately $1.08 per dollar invested. This is likely 
to be an underestimate, since it only captures two ecosystem services. 


58 Evaluation methods 
For valuation, we used a six-step procedure: 


1 Generate precise geospatial data for every site at which a GEF land deg- 
radation project operated. 

2 Use satellite information to derive long-term measurements of each of 
the three outcomes at each geographic location. 

3 Integrate the data generated in steps 1 and 2 with a wide set of geograph- 
ically varying ancillary data — such as nighttime lights, population, and 
distances to roads and rivers — to match project locations to “control” 
locations where no intervention occurred. 

4 Use a novel propensity score matching approach, causal trees, to examine 
the impact of project locations on each indicator of interest. 

5 Use observed patterns between these indicators and carbon sequestration 
to estimate the contribution of each project location in terms of tons of 
carbon sequestered. 

6 Apply a value transfer approach to enable users to valuate individual project 
locations alongside a presentation of reference values found in the literature.* 


Across the entire globe, within 25 km catchment areas, GEF land degrada- 
tion projects: 


e Increased normalized difference vegetation index (NDVI) by approxi- 
mately 0.03 (average NDVI is 0.55), 

e reduced forest loss by 1.3 percent (global mean of forest loss in all areas 
is 2.4 percent), and 

e increased the average size of forest patches by 0.25 km? (global mean size 
is 7.3 km’). 


The estimated carbon sequestered by the GEF was — on average — 
43.52 tC/ha. This equates to an estimated 108,800 tC sequestered by each 
land degradation project location. Across the 8,093 valuations of carbon 
identified as a part of the value transfer approach (Costanza et al. 2014) 
used to estimate project location valuations (deflated to 2014), we identi- 
fied a median dollar value of $12.90/t, drawing on academic, industrial, 
and government reports. Using this value, we estimated that GEF land 
degradation projects contributed $7.5 million (2014) on average to seques- 
tration alone — well above the average cost (approximately $4.2 million) of 
most such projects. 


Biodiversity impacts 


This analysis extended the VFM methodology we applied to the land degrada- 
tion case to GEF biodiversity projects, identifying their globally positive impact 
on vegetation productivity and forest cover (see Figure 4.4). Findings included: 


e Globally, GEF biodiversity projects tend to have a positive impact on 
both indicators assessed. 
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Figure 4.4 GEF biodiversity project outcomes 


e Performance improved as projects increased in size, with the strongest 
positive outcomes in the top 20 percent of funded projects. 
e Biodiversity projects had noticeable impacts after the first year of 


implementation. 


e Biodiversity projects are sensitive to access to electricity. 


We conducted valuation of biodiversity projects using the same approach 
as for land degradation activities (see Figure 4.5). Following this methodol- 
ogy, evaluators estimated a range of $60—-$166/ha of affected area for carbon 
sequestration with an additional value of $10—$41 attributable to soil retention 
benefits, for a total of $70-$207/ha. On average, we found a return of $1.04 
per dollar invested, although considerable uncertainty remains around this 
value. Geographically, impacts on forest cover were relatively homogeneous; 
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Figure 4.5 GEF biodiversity project location valuations 
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however, significant geographic heterogeneity existed in the case of vegeta- 
tion productivity. 


Measuring socioeconomic co-benefits 


Evaluations drawing on a variety of data sources and methods are particularly 
useful as we expand our analyses to better understand the socioeconomic 
co-benefits of the GEF’s environmental projects. Such approaches also help 
offset the biases and limitations of one method with the strengths of another. 

The IEO combined geospatial data with other data sources to measure 
socioeconomic co-benefits of GEF sustainable forest management (SFM) 
projects in Uganda. We expanded the VFM analysis to assess the impact 
and global environmental benefits of GEF investments and technical support 
through SFM interventions. This study assessed the impacts of SFM inter- 
ventions on environmental and biophysical variables, the co-benefits meas- 
ured in terms of socioeconomic indicators, and the estimation of monetary 
values of ecosystem services based on the principle of natural capital account- 
ing. The majority of GEF SFM project sites are in Sub-Saharan Africa, Latin 
America, and the Caribbean. Madagascar, Colombia, and Brazil have the 
largest number of GEF SFM project sites, which were in locations with very 
high initial conditions of deforestation. The GEF projects were targeted not 
toward areas that might maximize socioeconomic co-benefits, but those that 
were more likely to improve environmental outcomes. 


Effectiveness and valuation 


We estimated that the GEF SFM interventions® avoided approximately 
4,875 km? of deforestation over their respective implementation periods (an 
average of 2.5 km? per intervention location). Combined with improvements 
in vegetation density, this project cohort contributed additional sequestered 
above-ground carbon of 1.33 tC/ha/year, worth $727,990 annually on aver- 
age (under a conservative valuation of carbon at $12.90/Mt), compared to 
locations with no GEF interventions. This conservative estimate represents 
only the 1,924 project locations for which precise geospatial information was 
available. If valuation is extrapolated to cases for which such information 
was not available but a known site of implementation exists (3,585 inter- 
vention locations), the estimate is $1.36 million/year. This contrasts to an 
average implementation cost of $5.9 million, resulting in a break-even point 
of 4.5 years if only above-ground biomass is considered in valuation. 


Positive socioeconomic impacts 


A portfolio level global-scope analysis of economic and social co-benefits of 
GEF SFM projects suggest a small, positive impact on socioeconomic benefits 
indicated by nighttime light intensity. A majority of SFM interventions, espe- 
cially since 2014, were designed to address multiple focal area objectives. These 
interventions not only achieved the carbon sequestration described earlier, but 
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showed evidence of positive effect on nighttime lights, a proxy for economic 
development, that was not previously discernible. These findings may under- 
estimate the true impacts across the GEF SFM portfolio because they exclude 
projects for which the descriptions lacked high-precision geographic data. 

GEF SFM projects are also associated with an increase in household assets. 
The local-scope case study of Uganda (GEF IEO 2019b) provided more direct 
estimates of economic impacts, leveraging the World Bank Living Standards 
Measurement Survey (LSMS) to detect the impact of GEF projects on house- 
holds within 50 km. By matching LSMS locations proximate to GEF inter- 
ventions to those far away from GEF interventions, the local analysis showed 
that GEF SFM projects were associated with an increase in household assets 
between $163 and $353 (within 40-60 km, respectively). The Uganda case 
study shows that households within 50 km of a GEF implementation site 
tended to experience improvements in assets approximately $310 higher than 
those not proximate to a GEF implementation site. 

Despite their many advantages, particularly during crises such as that pre- 
sented by COVID-19, remote sensing and other forms of quantitative informa- 
tion do not address the “how” and the “why” behind the “what.” They do not 
provide an understanding of a program’s context, and they cannot explain the 
relationships and interactions among various factors that influence the complex 
systems within which GEF interventions take place. These methods need to be 
complemented by other forms of data collection such as interviews, case studies, 
and field verification to provide context for interpretation. As part of our the- 
matic and country evaluations, we collect such qualitative information through 
interviews, focus group discussions during field visits, case studies, and relevant 
documents. We use text analytics, qualitative comparative analysis, and rap- 
id-impact evaluation methods to understand the causal contribution of different 
conditions and factors. We have applied tools such as Nvivo to existing project 
documents to help identify a portfolio of interventions under specific themes of 
interest to extract information from documents and conduct analysis showing 
the linkages among interventions, outcomes, and contextual factors. 

A mixed-methods approach, applying a combination of quantitative and 
qualitative methods, and drawing on data from a variety of sources, has 
helped us offset the limitations of using these methods separately, particularly 
in explaining outcomes and influencing factors within a system of interest. 
By triangulating results from various methods, we can establish integrity, 
validity, and reliability of evaluation findings and provide sound evidence for 
conclusions and recommendations. 


Notes 


1. In other global development institutions, such as the World Bank and UNDP, 
operational staff conduct or manage terminal evaluations. The World Bank’s Inde- 
pendent Evaluation Group validates those terminal evaluations; the process varies 
across UN agencies. 
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2. For example, the World Bank brought out a practical guide based on extensive 
experience (Gertler et al. 2011), and, more recently, the Asian Development Bank 
issued a similar book with updated practices and material (White and Raitzer 
2017). 

3. According to GEF’s Project Management Information System, the GEF grant was 
$5.4 million. 

4. http://labs.aiddata.org/gef. 

. This estimate is based solely on the additive impact of GEF land degradation pro- 
jects on additional sequestration, that is, the total tons sequestered due to each GEF 
project that otherwise would not have been sequestered. This only includes esti- 
mates of gains due to changes along the three indicators examined (forest fragmen- 
tation, NDVI, and forest land cover) and thus may not represent the full envelope 
of all sequestration that is attributable to GEF projects. 

6. Subset of project implementation sites that met the inclusion criteria. 


on 
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5 Performance, effectiveness, 
and sustainability of 
GEF interventions 


Since its founding in 1991, the Global Environment Facility (GEF) has 
implemented projects and programs to enhance global environmental ben- 
efits in climate change, biodiversity, land degradation, chemicals and waste, 
and international waters. In fact, the GEF is the only public, international 
institution that addresses global environmental issues beyond climate change 
alone. More recently, the GEF has implemented projects that cut across focal 
areas (multifocal) and through a more integrated approach. These interven- 
tions are designed to deliver environmental benefits in more than one focal 
area and may also generate socioeconomic co-benefits. Across all GEF work, 
the projects and programs take into account the significance of gender, indig- 
enous peoples, and the important role of the private sector in the issues we 
address. As described in previous chapters, the GEF Independent Evaluation 
Office (IEO) exists to ensure through evaluation that GEF projects and pro- 
grams achieve their aims and make a lasting difference. To that end, we eval- 
uate the performance, effectiveness, and sustainability of the work. 

In this chapter, we first present the overall performance results of the GEF 
portfolio (GEF IEO 2017c) and the various factors driving these results. We 
look deeper into the results by exploring performance ratings by region and 
consider the attributes affecting performance: project design, quality of imple- 
mentation and execution, and cofinancing. We also evaluate the impacts of 
GEF policies, including those on gender and indigenous peoples. 

The second part of the chapter explores the longer term impacts and sus- 
tainability of GEF interventions and the channels through which these are 
achieved. Central to success for large-scale, long-term impacts is the concept 
of broader adoption, which occurs when governments and other stakeholders 
adopt, expand, and build on GEF interventions, based on initial success. 


Overall performance 


Satisfactory project outcomes 


Of the 1,706 projects completed since the GEF’s inception, 80 percent have 
had outcomes in the satisfactory range.' These ratings confirm almost 30 years 
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Figure 5.1 Projects with outcomes in the satisfactory range, by GEF period 


of GEF project success in steadily delivering expected short- to medium-term 
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results (see Figure 5.1). Our assessment of results draws primarily on data 
from the terminal evaluations submitted by the agencies and validated by 
the IEO. Most GEF-6 projects are still under implementation, thus terminal 
evaluations are not yet available. 


Project performance heterogeneity by regions 


In evaluating projects, we first consider groupings by region to provide a high- 
level understanding of project outcome success (Figure 5.2). The one region 
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Figure 5.2 Projects with outcomes in the satisfactory range, by region 
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that has a significantly lower percentage of projects earning a satisfactory rating 
(75 percent) is Africa. However, performance varies across subregions within 
Africa. In North African countries, outcomes of 90 percent of GEF projects 
were rated in the satisfactory range, while only 69 percent and 62 percent of 
the projects in East Africa and West Sub-Saharan countries, respectively, were 
rated in the satisfactory range.? This underscores the impact of the country 
context on development outcomes. Components of that context can include 


e Government ownership and support 

e Implementation capacity 

e Policy environment 

e Nongovernmental organization and community participation 
e  Private-sector participation 

e Economic and market conditions 


When we consider project outcomes by country groups, 87 percent of pro- 
jects implemented in the middle income countries, including Brazil, China, 
India, Mexico, and Russia, were in the satisfactory range, as compared with 
72 percent and 65 percent in the least developed countries and the small 
island developing states, respectively. 

Although several factors drive project performance, statistical analysis indi- 
cates a strong correlation between the quality ofimplementation and execution, 
level of cofinancing that ultimately materializes, and outcome performance. 
Weaknesses in project design contribute to poorer outcomes; these can include 
overly ambitious objectives, inadequate budgets for planned activities and 
arrangements to facilitate follow-up, weak institutional arrangements and 
government and stakeholder support, and poor monitoring and evaluation 
(M&E) design (GEF IEO 2014a). 


Implementation 


A project’s rating for quality of implementation reflects the extent to which 
the GEF has performed its role satisfactorily in project identification and 
preparation, start up, supervision, application of the GEF policies and pro- 
cedures, and project M&E. It also reflects the extent to which the GEF took 
timely actions to correct any identified gaps in project design and imple- 
mentation. National executing agencies, which can be either governmental 
or sometimes nongovernmental, are typically responsible for carrying out 
the project activities on the ground under the supervision of the GEF. These 
activities include implementing project design, procurement, stakeholder 
consultations, and project monitoring. Parallel to the increase in overall 
project satisfaction, the quality of implementation of GEF projects has also 
improved over time. Overall, 81 percent of projects received satisfactory rat- 
ings in project implementation, with the performance improving over the 
course of the GEF periods (Figure 5.3). 
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Figure 5.3 Projects with implementation rated in satisfactory range, by GEF period 


Another crucial component of project performance is raising additional 
financing from governments or the private sector. This demonstrates com- 
mitment to a project and supports the long-term financial sustainability of 
interventions. The GEF Co-Financing Policy (GEF 2014a),° operational- 
ized during GEF-6, targets a 6:1 level of cofinancing for the overall GEF 
portfolio. On average, cofinancing of $6.49 per dollar materialized in that 
period. In 61 percent of the projects completed in the GEF’s three decades, at 
least 90 percent of the promised cofinancing actually materialized. Realized 
cofinancing is higher in China, Brazil, India, Mexico, and the Russian 
Federation, at 82 percent, compared with 56 percent in least developed coun- 
tries and 67 percent in the small island developing states, where raising extra 
resources from the government or the private sector is a bigger challenge. 
Meeting cofinance targets is a challenge in Africa, which is reflected in the 
lower project outcome ratings for that region. 

Project monitoring involves the design and implementation of an M&E 
plan by GEF Agencies to track implementation progress and achievement 
of results. An M&E plan specifies indicators to track processes and results, 
responsibilities, frequency of data collection, reporting procedures, and a 
budget for monitoring activities. The plan may need to be updated or modi- 
fied during implementation. Because the quality of M&E design and imple- 
mentation influences project performance, our rating reflects the extent to 
which an M&E plan was well designed and well implemented. (See Chapter 6 
for more discussion of M&E design and implementation.) Overall, 67 per- 
cent of GEF projects have received a satisfactory rating for M&E design, 
with clear improvement occurring over GEF periods (Figure 5.4). However, 
ratings have generally been lower for overall implementation of M&E plans 
than for design since GEF-3. 
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Figure 5.4 Project M&E design and implementation rated as satisfactory, by GEF period 


Sustainability outcomes: achievements and limitations 


Donors and funding recipients are interested in achieving project outcomes, 
but are even more concerned with long-term sustainability of these out- 
comes. Sustainability is influenced by the quality of project preparation, 
country context, government support, quality of implementation and exe- 
cution, M&E, and the actual amounts of realized cofinancing (GEF IEO 
2018a). An analysis of terminal evaluations for GEF projects shows improve- 
ment in ratings for sustainability of outcome across the GEF replenishment 
periods (see Figure 5.5); 62 percent of the completed GEF projects were 
rated in the “likely” range for outcome sustainability. However, this means 
that roughly 4 out of 10 projects face considerable risks to continuation of 
their benefits. Among the regions, Africa has a significantly lower percentage 
of projects that are rated as likely to be sustainable, reflecting differences in 
regional and country capacities (see Figure 5.6). Within Africa, countries 
in North Africa have a higher percentage of sustainable projects (64 per- 
cent) than those in East Africa and West Sub-Saharan Africa (35 percent).* 
Considering sustainability of projects in select country groups, 85 percent 
in China, Brazil, India, Mexico, and the Russian Federation were rated in 
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Figure 5.5 Projects with likely sustainability, by GEF period 


the likely range for sustainability compared to 44 percent in least developed 
countries. Projects in landlocked developing countries and small island devel- 
oping states closely tracked the portfolio performance for sustainability, at 
58 percent and 64 percent. 

These findings on sustainability are not unique to the GEF — the percentage 
of GEF completed projects with a likelihood of sustainability at project com- 
pletion is comparable with other multilateral organizations. These organiza- 
tions have differences in terms of their mandate, geographical coverage, and 
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Figure 5.6 Projects with likely sustainability, by GEF region 
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scale of activities, but their percentages of projects rated in the likely range 
for sustainability are within a 12-point range: 


e African Development Bank: 52 percent 

e Asian Development Bank: 66 percent 

e International Fund for Agricultural Development: 60 percent 
e Inter-American Development Bank: 62 percent 

e World Bank: 65 percent 

e GEF: 62 percent 


The only outlier is the United Nations Environment Programme, where the 
percentage of projects with likely sustainability is 79 percent.” 


Sustainability beyond project closure 


To fully grasp whether a GEF project’s outcomes are sustainable, an evalua- 
tion at the time of closure may be too soon. For most GEF projects, outcomes 
are sustained during the post-completion period. In fact, a high percent- 
age of projects achieve environmental stress reduction and broader adoption 
after the project is completed, rather than during implementation. This sug- 
gests that achieving and observing sustainability takes time, often extending 
beyond the project implementation period. The key factors that contribute to 
this sustainability after completion include high stakeholder buy-in, political 
support, availability of financial support for follow-up, and sustained efforts 
from the executing agency. 

The closure of a project may be too early for project outcomes to mani- 
fest fully: a change in a protected area’s environmental status can take more 
than a decade, and policy measures or new technologies implemented during 
projects also may achieve outcomes past closure. For example, stakeholder 
involvement in the Conservation and Sustainable use of the Mesoamerican 
Barrier Reef System Project led to an assessment by the United Nations 
Educational, Scientific, and Cultural Organization two years after the 
project’s completion, resulting in the official placement of the Belize Barrier 
Reef World Heritage Site on the list of endangered sites. Another project, 
the bikeshare system introduced in Rosario, Argentina, by the Latin America 
and the Caribbean Regional Sustainable Transport and Air Quality Project, 
had higher usage rates three years after project completion than at the com- 
pletion date, with rates expected to increase further with planned expansion. 
Box 5.1 presents details on outcome sustainability for a project in the Ba Be 
Protected Area. 


Implementation of GEF policies 


The GEF Policy on Gender Equality (GEF 2017a) has advanced the GEF’s efforts 
to strengthen gender mainstreaming in GEF programming and operations in 
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Box 5.1 


Outcome sustainability in Ba Be Protected Area 


The GEF project Promotion of Sustainable Forest and Land Management in the 
Vietnam Uplands, implemented from 2010 to 2013, supported sustainable forest 
management in Ba Be National Park. Threats to the forest include fragmentation 
of habitat, conversion of forest land for infrastructure and agriculture, illegal hunt- 
ing, and overuse for non-timber forest products. The project was focused on local 
communities and piloted several approaches: 


* improved systems for animal husbandry and conservation of sloping lands 

e bio-energy applications 

* payment for ecosystem services including participatory approaches to forest 
protection 

* eco-tourism 


Remote sensing data showed that forest cover in Ba Be Protected Area has 
remained stable after project completion due to the community-focused project 
design, attention to income-generation activities, and sustained support from pro- 
vincial governments (see Figure 5.7). 
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Figure 5.7 Forest cover loss of Ba Be 


a systematic manner, and there is further room for improvement in implemen- 
tation. Since the policy’s implementation in 2017, project documentation has 
shown that gender consideration at the point of chief executive officer project 
endorsement/approval rose from about 57 percent to almost 98 percent. The 
GEF Gender Partnership is slowly developing into an effective platform on 
which to build a wider constituency on gender and the environment, provid- 
ing a forum for leveraging the broad range of member skills and experiences 
on gender equality and women’s empowerment. 
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OPS6 identified specific shortcomings and aspects for improvement in this 
policy. First, it stops short of providing a compelling rationale for why gen- 
der matters in environment-focused interventions. It also does not provide a 
rationale as to how the inclusion of gender equality in environmental projects 
would generate benefits beyond effectiveness and efficiency. The policy could 
reference the gender-related mandates or decisions of the five conventions 
the GEF serves. Although gender performance in GEF interventions has 
improved since the introduction of the policy, only about 14 percent of pro- 
jects at entry included a gender analysis, which is integral to mainstreaming. 

The GEF policies and guidance on safeguards and indigenous peoples have 
advanced the GEF’s efforts in these areas; however, gaps exist in the policy 
frameworks relative to good practice in partner agencies and in implemen- 
tation. The adoption of the GEF Policy on Agency Minimum Standards on 
Environmental and Social Safeguards in 2011 prompted several agencies to 
develop or revise their own safeguard systems. By design, these improve- 
ments have occurred principally during the accreditation process for new 
agencies and compliance review for existing agencies. Gaps exist in the 
framework in relation to recent updates made in GEF partner agencies, and 
no guidance addresses ongoing reporting or monitoring on safeguard-related 
issues during project implementation. Most GEF Agencies comply with the 
obligations specified under GEF Minimum Standard 4: Indigenous Peoples. 
These principles and guidelines reinforce GEF policies toward indigenous 
peoples, but lack practical guidance on project design and indicators, or a list 
of requirements that could aid in operationalizing the minimum standard 
and other relevant GEF policies. 


Focal area performance 


In the course of the OPS6, we considered the outcomes and sustainability of 
the projects in each of the GEP’s five focal areas and multifocal projects. This 
section explores each of the focal areas: its definition, project success rates 
on various metrics and across regions, and progress in broader adoption and 
mainstreaming. This takes place when information, lessons, or specific results 
of GEF interventions are incorporated into broader stakeholder mandates and 
initiatives such as laws, policies, regulations, or programs. We also consider 
the focal areas’ success in achieving catalytic effects through the steps on the 
catalytic chain: production of a public good, demonstration, replication, and 
scaling up (see Figure 5.8). 


Biodiversity 


Changing the trajectory of biodiversity loss means addressing its five primary 
drivers: habitat change, overexploitation or unsustainable use, invasive alien 
species, climate change, and pollution. All of these are intensifying, particu- 
larly habitat loss driven by the expansion of agriculture. 
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Figure 5.8 Focal area outcomes and sustainability ratings 


The GEF’s biodiversity strategy aims to maintain globally significant 
biodiversity in landscapes and seascapes. To achieve this, the GEF provides 
financial resources for developing countries and countries with economies in 
transition to implement the Convention on Biological Diversity. GEF invest- 
ments in this focal area focus on: 


e Mainstreaming biodiversity across sectors, landscapes, and seascapes 
¢ Addressing direct drivers to protect habitats and species 
¢ Further developing biodiversity policy and institutional frameworks 


Our evaluation of the biodiversity portfolio using 554 terminal evaluations 
found that 83 percent of projects in this focal area had satisfactory outcome 
ratings. The biodiversity portfolio performed slightly better than the overall 
GEF portfolio on all counts except sustainability; 57 percent of biodiversity 
projects were rated in the likely range for sustainability. When considering 
this focal area by region, we found that projects in Eastern Europe, Central 
Asia, and global projects tend to have the highest ratings on all indicators, 
while projects in Africa have the lowest ratings for outcomes, sustainability, 
M&E design and implementation, and quality of project implementation and 
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execution. In all regions, average execution quality and average implementa- 
tion quality are rated substantial to high, but overall sustainability of projects 
is rated low to modest. 


Protected areas and protected area systems 


Most GEF biodiversity interventions focus on strengthening protected areas 
and protected area systems and ensuring their sustainability through strate- 
gic expansion, improved management, sustainable financing, and targeting 
factors in biodiversity loss beyond the protected areas. These interventions 
have contributed to preserved habitats, positive trends in species population, 
and reduction of threat. Of the 1,292 protected areas supported by the GEF, 
58 percent are classified as key biodiversity areas, 31 percent do not hold that 
classification but have received international designations of high biodiversity 
and/or cultural value,’ and the remaining 11 percent are of local or national 
significance. This points to the high relevance of GEF investments in areas 
with significant biodiversity. 

Our analysis methods demonstrate how GEF support contributes to bio- 
diversity conservation by helping reduce habitat loss in protected areas. A 
geospatial analysis of data between 2001 and 2012 showed that forest loss in 
GEF-supported protected areas was half the loss in those without GEF sup- 
port in the same biomes and countries. The results also showed that countries 
with long-term GEF support had better conservation outcomes. Through 
targeted interventions at the site level, GEF-supported protected areas gener- 
ally see positive trends in species populations and reduced pressures to biodi- 
versity, such as from agriculture, cattle ranching, and tourism. Overall, GEF 
support contributed to change in key factors affecting biodiversity conserva- 
tion in protected areas — such as building stronger management capacities, 
promoting participatory planning and stakeholder support, improving tech- 
nical capacities, mainstreaming biodiversity and sustainable use, and demon- 
strating social and economic benefits. Box 5.2 presents an illustrative case for 
Mexico. 

Although determining the returns to GEF interventions has often been a 
challenge, it has always been a priority area for donors. The IEO piloted an 
approach to determine the efficiency or “value for money” of GEF projects 
using a value transfer approach, which uses natural capital accounting to esti- 
mate the value of land degradation and biodiversity in terms of the amount 
of carbon sequestered. We applied this approach to 550 GEF biodiversity pro- 
jects across 3,095 project locations. The analysis estimated the impacts along 
multiple indicators to capture changes in natural capital in three ecosystem 
services: carbon sequestration, recreation, and soil retention. We found that 
GEF biodiversity projects generate positive returns on investment across 
the board. The overall value of a GEF biodiversity project was estimated at 
$6,065.59/ha. On average, it generated a return of $1.04 per dollar invested, 
a figure that is likely an underestimate. 
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Box 5.2 


Mexico: assessing GEF support’s impact on forest preservation 


In Mexico, the GEF has supported the National System of Natural Protected 
Areas (SINAP) for nearly 25 years through three projects. The first was originally 
intended to strengthen protected area management in up to 17 Mexican reserves. 
It was restructured to include a flexible endowment for long-term funding to 
support high-quality staffing and operations in 10 protected areas. It also provided 
funds for workshops and knowledge exchanges. Over time, this model of learn- 
ing by doing and exchange of knowledge led to the strengthening of Mexico’s 
National Commission on Natural Protected Areas (CONANP), an institution 
now highly respected in the country’s public administration system. In 2008, the 
Mexican government decided to bring all CONANP staff under the government 
budget, and made available endowment funds previously dedicated to supporting 
23 protected areas to civil society organizations implementing strategic projects 
in these areas. Thus, GEF support to Mexico’s protected area system was main- 
streamed through strengthened government institutions. 

Given the GEF’s long-term support to Mexico and fewer gaps in identifying 
GEF-supported protected areas, we conducted a robust, quasi-experimental anal- 
ysis to assess the impact of GEF funding. Using propensity score matching, we 
used satellite data products to compare GEF-supported protected areas with simi- 
lar protected areas that did not receive GEF support. We found that, from 2001 to 
2012, GEF-supported areas in Mexico avoided up to 23 percent more forest loss 
than those that did not receive direct GEF support. 


(Source: GEF IEO 2016a) 


Projects that have mainstreaming with other components or mainstream- 
ing alone had more sustainable and more satisfactory outcomes than those 
focused exclusively on protected areas. Box 5.3 presents an example in India. 
Eighty-nine percent of the biodiversity mainstreaming projects with the 
combination of forestry and agriculture have satisfactory ratings, as do 86 
percent of the projects in the agriculture sector. In comparison, 56 percent 
of fisheries projects, which do not address mainstreaming, have satisfactory 
outcome ratings. In terms of sustainability, 90 percent of the projects in the 
agriculture sector, which incorporate mainstreaming, were rated satisfactory, 
as compared to 50 percent in fisheries. 


Climate change 


Action on climate change requires ambitious programs to limit emissions of 
greenhouse gases while supporting communities’ adaptation to the unavoid- 
able impacts of existing climatic changes. It also means embracing the poten- 
tial of the green economy that balances economic, social, and environmental 
priorities. 
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Box 5.3 


India: demonstrating impacts in the East Godavari 
River Estuarine Ecosystem 


Habitat destruction, pollution, and overexploitation of coastal and marine resources 
pose major threats to the biologically and economically important East Godavari 
River Estuarine Ecosystem (EGREE). The EGREE includes the Coringa Wildlife 
Sanctuary, the second largest extension of mangroves on the eastern coast of India. 
It is also a fast-growing development hub, including manufacturers, industries, 
and offshore oil and gas exploration. EGREE ecosystem services directly provide 
livelihoods to around 100,000 people in the 44 villages surrounding the sanctu- 
ary. Major activities include fisheries, aquaculture, and agriculture. The Godavari 
estuary, comprising 62,000 ha, lost 1,250 ha of mangroves in between 1992 and 
2004, primarily due to anthropogenic pressures (Satapathy et al. 2007). 

The Mainstreaming Coastal and Marine Biodiversity Conservation into 
Production Sectors in the East Godavari River Estuary, Andhra Pradesh child 
project, aimed to enable a governance environment that would prevent further 
degradation of coastal and marine ecosystems, allowing the continuous flow of 
ecosystem goods and services — such as coastal protection and fisheries — while 
preserving an ecosystem of unique biological value. The approach included 
mainstreaming biodiversity into public- and private-sector plans and developing 
cross-sector institutional mechanisms to harmonize development and conserva- 
tion of biodiversity. 

We used data derived from daily satellite observations to examine the long-term 
spatial and temporal patterns of vegetation to assess the project’s impact on the 
local ecosystem. The results suggest no net loss of vegetation cover in the project 
area. In fact, we estimated an overall minor increase in mangrove density when 
comparing the project period 2011-2015 to the pre-project period 2007-2009. 


(Source: GEF IEO 2017a) 


The GEF serves as part of the Paris Agreement’s financial mechanism, 
helping to ensure transparency and assisting countries in meeting their goals. 
The GEPF’s strategies address both climate change mitigation (supporting 
developing countries’ transformational shifts toward low-emission develop- 
ment pathways) and climate change adaptation (supporting climate-resilient 
development pathways while reducing exposure to the immediate risks posed 
by climate change). 

The GEP’s strategies for climate change mitigation programming focus 
on promoting innovation and technology transfer and creating an ena- 
bling environment. Approximately 77 percent of completed projects in 
the climate change mitigation portfolio have overall outcome ratings in 
the satisfactory range, with some themes performing better than others. 
Sustainability remains a challenge; these ratings were linked to projects’ 
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financial stability and cofinancing did not fully materialize for nearly three 
quarters of these projects. 

The GEF supports scaling-up of public and especially private climate invest- 
ment by strengthening the enabling environment. Capacity-building and 
policy activities can leverage significant impact, according to our 2014 eval- 
uation of GEF climate change mitigation support in China, India, Mexico, 
and Russia (GEF IEO 2014b). GEF climate change projects frequently focus 
on reforming policy and regulations, building public- and private-sector 
capacity, and raising awareness to reduce information barriers and support 
market change. GEF support has been limited but critical for development 
of some countries’ energy policies and laws, primarily regarding energy 
efficiency and renewable energy. For successful private-sector engagement, 
technical assistance and capacity building are critical, particularly in projects 
piloting financial models to scale up energy efficiency and renewable energy 
adoption. These lessons are significant in light of the climate change focal 
area’s substantial private-sector portfolio. 

Most GEF climate change projects have shown some evidence of catalytic 
effects. The most common evidence was mainstreaming (primarily through 
policy or regulatory reform) in about 70 percent of closed projects. Performance 
was less strong for replication, scaling up, and market changes. The most 
impactful GEF mitigation projects in China, India, Mexico, and Russia are 
those using comprehensive approaches to address market barriers and specif- 
ically targeting supportive policy frameworks. Success in scaling up typically 
involved follow-on funding from the GEF or other multilateral or bilateral 
donors, or support for nationally owned programs via projects with significant 
multilateral development bank (MDB) cofinancing. We found similar catalytic 
effects in the Least Developed Countries Fund and Special Climate Change 
Fund (LDCF/SCCF) portfolio, where virtually all 27 completed projects 
achieved production of a public good and demonstration, but performance was 
not as strong in replication and scaling up (GEF IEO 2016b, 2017b). 

Typical GEF support in this focal area includes aid to countries in meet- 
ing their convention obligations; flexible grant financing; focus on the ena- 
bling environment to support scaled-up climate investment; emphasis on 
demonstrating technologies and financial approaches, including innovative 
and risk-sharing approaches; and the ability to fund integrated projects across 
environmental issues. Other potential niches for the GEF include focusing 
on upstream activities to develop supportive conditions through capacity 
building, technical assistance, and policy and regulatory reform to accelerate 
market development. The GEF may also contribute by piloting innovative 
technologies and market mechanisms. 


Climate change adaptation and the LDCF/SCCF 


The interventions of the 297 projects in the LDCF/SCCF portfolio are 
highly relevant to decisions by the United Nations Framework Convention 
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on Climate Change, GEF objectives, and the GEF strategic pillars for climate 
change adaptation: 


e Reduce vulnerability and increase resilience through innovation and 
technology transfer for adaptation 

e Mainstream adaptation and resilience for systemic impact 

e Foster enabling conditions for effective and integrated adaptation 


Our quality-at-entry review showed that 98.4 percent of LDCF/SCCF- 
funded projects have a high to very high probability of delivering tangible 
adaptation benefits. Of 27 completed LDCF/SCCF projects, those focused 
on farming were most typical, such as introducing and demonstrating 
drought-resistant crop varieties. Projects’ primary constraint to scaling up 
was securing sufficient resources after implementation and/or mainstreaming 
the work, such as within national budgets. Among the complete projects, 75 
percent were rated likely to achieve sustainable outcomes, while the remain- 
ing projects were rated moderately unlikely. 

Despite the continued relevance of the funds and evidence that projects 
are delivering tangible adaptation results, LDCF/SCCF resources have been 
completely inadequate to meet demand. Unpredictability of funding cre- 
ates uncertainty for GEF Agencies and countries that rely on LDCF/SCCF 
support to implement their primary climate change adaptations. It also neg- 
atively influences stakeholders’ perception of the funds’ transparency and 
affects the funds’ efficiency. 


Chemicals and waste 


Present in all of Earth’s ecosystems, toxic chemicals affect biodiversity, agri- 
cultural production, water quality, and human health. Numerous interna- 
tional agreements address the production, use, and disposal of chemicals, but 
all three are rapidly increasing in developing countries and countries in eco- 
nomic transition. These rapid changes increase economic opportunities but 
must be matched by enhanced initiatives for sound chemicals and waste man- 
agement to prevent significant harm to humans, wild species, and ecosystems. 
The GEF has two strategic objectives in this focal area: 


1 Develop the enabling conditions, tools, and environment for the sound 
management of harmful chemicals and wastes 

2 Reduce the prevalence of harmful chemicals and waste and support the 
implementation of clean alternative technologies/substances 


Ambitious Sustainable Development Goal targets related to the environ- 
mentally sound management of chemicals and waste make this focal area 
increasingly important. Its projects adhere to the guidance of the conven- 
tions for which it is the financial mechanism and support the goals of related 
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multilateral environmental agreements, including the Strategic Approach to 
International Chemicals Management, the Basel and Rotterdam Conventions, 
the Montreal Protocol, and the Minamata Convention. 

Projects in this focal area have largely performed on par with those in 
other focal areas in terms of achievement of outcomes: among 23 closed pro- 
jects, 79 percent of national projects and 80 percent of global projects have 
satisfactory outcomes. By region, success rates were higher in Asia (91 per- 
cent) and Europe and Central Asia (79 percent), and lower in Latin America 
and the Caribbean (67 percent) and Africa (50 percent). Performance data 
indicate potential challenges for chemicals and waste projects with regard 
to the sustainability of persistent organic pollutants (POPs) results and the 
outcomes, sustainability, and quality of implementation of multicountry pro- 
jects. Seventy-four percent of the projects demonstrated strong country own- 
ership, which is one driver of success among projects addressing POPs; the 
other is private-sector commitment. 

More than 80 percent of chemicals and waste projects engaged the pri- 
vate sector, primarily large national and multinational corporations, in cofi- 
nancing, capacity building, direct support, or participation in project design 
and implementation. One ongoing challenge is a deficiency of incentives, or 
sometimes scope, to combine chemicals-related issues to promote sector-wide 
approaches, such as updating legislation to fully address chemicals and waste, 
rather than just PCBs, or to address solid waste management more broadly, 
rather than just POPs waste. Some multifocal area projects — notably, the 
Sustainable Cities Integrated Approach Pilot (IAP) Program — focus broadly 
on solid waste management, with benefits to climate change mitigation and 
other toxic substances. This challenge affects scaling up; broader institutional 
infrastructure may be needed to support hazardous waste or chemicals man- 
agement. This challenge also can affect cofinancing or mainstreaming into 
larger investment projects. 

Balancing hard outcome targets (such as tons of POPs and mercury dis- 
posed) against the importance of soft activities and outcomes (such as support 
for developing policy and regulatory frameworks and institutional strength- 
ening) is difficult. In our evaluation, many interviewees noted the lessons 
learned from the Montreal Protocol in terms of the value of strong regulatory 
regimes to support ozone-depleting substance phaseout and ensure that the 
private sector continues to implement the good practices that individual pro- 
jects demonstrated. A related challenge is the trade-offs sometimes necessary 
between hard outcome targets and political realities, such as tackling the 
biggest problem sites to meet convention targets versus prioritizing countries 
that may not have yet received funding for their national implementation 
plan (NIP) or funding lower tonnage projects in Africa. 

The most common form of broader adoption among chemicals and waste 
projects was mainstreaming, with limited success in scaling up or replication. 
Mainstreaming was achieved primarily through the adoption and enforce- 
ment of laws and regulations at national and local levels. Until recently, 
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projects have not sufficiently focused on approaches to scale up or replicate 
successes. Many completed projects have demonstrated the collection and 
destruction of POPs and reduced environmental stress in a relatively straight- 
forward manner but have not established sustainable strategies and financial 
mechanisms to scale up those results. Among the few exceptions are several 
GEF POPs interventions in China. For example, the project Improvement 
of DDT-based Production of Dicofol and Introduction of Alternative 
Technologies including IPM for Leaf Mites Control in China finalized an 
integrated pest management national replication program prior to project 
completion. Replication activities have been initiated at several provincial 
locations and are expected to expand nationwide and to cover additional 
crops. Factors in the success of this program are availability of financial 
resources and technology support. 

The GEF cannot finance the collection and destruction of every ton of leg- 
acy POPs, nor can it fund the conversion of every industrial facility to cleaner 
production processes. A more robust theory of change is needed for how the 
GEF’s demonstration activities will catalyze broader action and impact in the 
chemicals and waste focal area. This may involve the development of innova- 
tive private-sector partnerships, economic instruments, and financial models, 
as envisioned in the GEF-6 Chemicals and Waste Focal Area Strategy. 


International waters 


While not serving one specific international agreement, the international 
waters focal area contributes to the enhancement of regional security and 
supports the sustainable use and protection of transboundary waters, their liv- 
ing resources, and dependent ecosystems. Projects in this focal area also help 
ease tensions between riparians, improve livelihoods of the vulnerable, and 
sustain economic and social development, consistent with the 2030 Agenda 
for Sustainable Development. 

The international waters focal area follows a stepwise, long-term, eco- 
system-based approach to building transboundary cooperation and the res- 
toration and protection of transboundary water bodies. With a reliance on 
science and knowledge management and a systemic view of the many inter- 
connected variables controlling water, this approach places the focal area in a 
unique position as a catalyst of integration. International waters foundational 
projects have demonstrated that solutions to water concerns lie not just in 
improving water supply and treatment, or in protecting aquatic ecosystems 
and environmental flows, but also — and often primarily — in distant sectors 
such as food and energy production, trade, land use and urban planning, 
industrial processes, and forests management. 

Among 127 completed projects related to international waters, 79 percent 
of regional projects have satisfactory outcomes, as do 64 percent of national 
projects. On a regional basis, success rates were highest in Asia (80 percent) 
and lowest in Europe and Central Asia (65 percent). 
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A unique feature of this focal area is the prevalence of projects (11 percent, 
receiving 6 percent of focal area funding) directed at learning, improving 
project quality, capturing existing knowledge on water issues, assessing global 
international waters priority concerns, and making available to all project 
knowledge and experiences. Through building databases, innovating knowl- 
edge management, thinking across jurisdictions and focal areas, and linking 
freshwater with marine environments, this focal area has achieved significant 
outcomes in learning from its own experiences and in systematic exchange 
among projects and partners. The GEF has had considerable success in pro- 
moting transboundary cooperation around freshwater basins and is the only 
multilateral fund in this space. Solutions to transboundary water concerns 
require national actions in multiple dimensions and GEF focal areas. Through 
its ecosystem approach and transboundary diagnostic analysis—strategic action 
program consensus-building process, the international waters focal area pro- 
vides countries with the framework to direct part of their investments of GEF 
System for Transparent Allocation of Resources (STAR) funds where they are 
most needed to balance transboundary water uses. 


Land degradation 


Land degradation — the deterioration or loss of the productive capacity of the 
soils for present and future use — is a global challenge and a pressing environ- 
mental problem that will worsen without rapid remedial action. It contrib- 
utes significantly to climate change and reduced productivity of croplands 
and rangelands worldwide. The GEF works to arrest and reverse desertifica- 
tion and deforestation across sectors from crop and livestock production to 
water resource management. 

The focal area strategy closely reflects convention guidance and, since 
2016, has integrated into its programs and priorities the United Nations 
Convention to Combat Desertification’s (UNCCD’s) objective of achieving 
land degradation neutrality.” Although land degradation problems are most 
obvious in desert margins and other arid lands, GEF support for tackling 
land degradation continues to pursue geographical balance and to include 
non-dryland areas. The land degradation focal area addresses unsustainable 
land management practices and degradation issues as driven by country pri- 
orities and needs, going beyond arid, semiarid, and dry subhumid areas. 

Integrated landscape projects have increased, but restoration activities are 
limited. Agricultural lands, rangelands, degraded productive lands, and desert 
lands are the most frequent land types in single focal area projects, with fewer 
projects addressing urban lands. Since the early 2000s, projects have shifted 
toward more holistic, integrated landscapes with an almost 30 percent increase 
in integrated landscapes over that time frame. Most GEF efforts are focused 
on sustainable land management, and restoration is slowly gaining attention. 

Performance of this focal area portfolio is slightly lower than the GEF 
average, with satisfactory outcome ratings for 76 percent of projects. Land 
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degradation focal area investments have led to positive impacts on UNCCD 
targets, specifically reducing forest loss and forest fragmentation, and increas- 
ing vegetation productivity. A geospatial impact analysis and a value-for- 
money analysis showed important reductions in fragmentation and forest loss 
and an increase in vegetation productivity and carbon sequestration. 

Improvements in vegetation cover from land degradation projects also gen- 
erated environmental benefits through higher levels of carbon sequestration. 
Based on our value-for-money analysis, an estimated average of 108,800 tC 
is sequestered in each land degradation project location. At a valuation of 
$12.90/t (GEF IEO 2018d), individual land degradation projects contributed 
$7.5 million on average to sequestration — well above the average cost of most 
of these focal area projects. 

GEF land degradation projects yield overall positive returns on investment. 
Between carbon sequestration and soil retention values, we estimate bene- 
fits from a single focal area land degradation project at $62-$186/ha. After 
accounting for costs, the per-dollar return on investment for land degrada- 
tion projects is approximately $1.08 per dollar invested, and this is likely an 
underestimate because it captures only two ecosystem services. 

Project duration, infrastructure access, and initial conditions are correlated 
with effectiveness. A time lag of 4.5-5.5 years was an important inflection 
point at which we observed impacts. This highlights the length of time it 
takes to observe actual environmental improvements on the ground, a factor 
that poses challenges both to evaluators and to project proponents expecting 
quick solutions. Projects with access to electricity tend to have some of the 
largest relative positive impacts, potentially due to better infrastructure and 
access to energy sources for irrigation. Understandably, GEF projects tended 
to have a larger impact in areas with poorer initial environmental conditions. 
However, this does not necessarily mean that investments should focus on the 
most degraded areas, because priority areas for restoration may lie elsewhere. 

Developing multistakeholder partnerships, improving income generation, 
and addressing climate risks are important for project success in this focal area. 
Evidence from case studies demonstrates the value of effective partnerships 
between government agencies, civil society, the private sector, and grassroots 
organizations in addressing policy issues such as land tenure rights, environ- 
mental issues such soil erosion, and loss of land productivity. Project activities 
that focus on income and market access — and that teach adaptive practices to 
cope with climate risks — improve both environmental and socioeconomic 
outcomes and influence residents’ decisions not to migrate to urban areas. 


Multifocal area projects 


Multifocal area programs and projects are funded through allocations from 
more than one focal area, or are labeled as such. The 250 multifocal area 
projects account for 10 percent of the GEF portfolio, equivalent to 13 per- 
cent of total GEF grants. The most common combinations in multifocal area 
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projects include biodiversity and land degradation (54 percent); half of these 
also include climate change. 

The great majority of multifocal area projects adhere to convention guid- 
ance and address both global and national priorities by targeting environ- 
mental and socioeconomic objectives together. The multifocal area portfolio 
reflects global trends toward integration across sectors and between environ- 
mental and socioeconomic goals stated in the three Rio Conventions and the 
Sustainable Development Goals (SDGs). Multifocal area projects also respond 
to national priorities with flexibility in jointly addressing global environmen- 
tal commitments and national sustainable development goals through inte- 
grated approaches including integrated ecosystem management, integrated 
landscapes, protected area systems, and production landscapes. These pro- 
jects allow countries to achieve multiple focal area and livelihood objectives 
simultaneously. 

Among completed multifocal area projects, 77 percent were rated mod- 
erately satisfactory or higher. The large majority reported interconnected 
environmental and socioeconomic benefits and broader adoption of interme- 
diate outcomes. The most common positive environmental outcomes were 
reduction in environmental stress or threats (90 percent) and improvements 
in ecosystem cover or quality (71 percent). Increased income or greater access 
to capital was the most frequently reported socioeconomic outcome (79 per- 
cent). Eighty percent of projects reported that broader adoption had taken 
place or begun by project end, primarily as mainstreaming and sustaining 
of outcomes, and replication. Engagement of key stakeholders, good project 
design, and coordination with related initiatives were among the factors most 
frequently cited as contributing to success. Poor achievement of outcomes 
and adoption in multifocal area projects were linked primarily to low insti- 
tutional capacity among executing agencies. 

Multifocal area projects have the potential for producing synergies and mit- 
igating trade-offs. Synergy refers to multiple benefits that are achieved either 
simultaneously through a single intervention or through the interaction of 
outcomes of at least two interventions. A trade-off is defined as a reduction 
to one benefit in the process of maximizing or increasing another benefit. 
We identified synergies across focal areas and socioeconomic priorities in tree 
planting, clean energy technologies that reduced fuelwood use, sustainable 
agriculture practices such as the use of organic waste as fertilizer, and ecosys- 
tem protection and rehabilitation. Trade-offs were identified between envi- 
ronmental and socioeconomic benefits, between objectives within or among 
focal areas, between short- and long-term objectives, between beneficiaries, 
and between local and national benefits. Three types of mitigating measures 
have reduced potential losses from trade-offs: compensation, compromise, 
and value addition. Compensation involves direct payment or replacement 
of income to address the loss of socioeconomic benefits. Compromise occurs 
when the benefit to one focal area was decreased to reduce the anticipated 
loss to another focal area or socioeconomic aspect. Value addition occurred 
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when an intervention not only addressed the trade-off, but also created focal 
area and socioeconomic benefits beyond the status quo, essentially producing 
synergies. We found that projects generate more synergistic benefits when 
the set of interventions serves multiple objectives; involves multiple sectors 
in decision-making; or is delivered through integrated spatial units, such as a 
village, landscape, or watershed. 

Multifocal area projects also have efficiency trade-offs. They can create 
opportunities to fulfill global and national commitments simultaneously, lev- 
erage focal area funding, streamline project management costs, and increase 
multisectoral interaction. The option to integrate funds from multiple focal 
areas has allowed each focal area’s priorities to be addressed in more inter- 
ventions while using less of each focal area’s allocation, especially for the 
land degradation focal area, which typically receives much lower funding. 
Although the involvement of more actors at all levels makes the project design 
and approval more complex, it provides valuable interaction among sectors. 
The typically larger size of multifocal area projects also allows economies of 
scale in project management. 

Costs are incurred with losses of efficiency, mainly during project design, 
review, and monitoring, due to the greater number of stakeholders and sec- 
tors involved. Multifocal area projects require more focal area expertise and 
agreement among stakeholders in their design and approval. Whether at the 
country or corporate level, the involvement of more actors leads to more 
complex and time-consuming decision-making. In some cases, this has cre- 
ated competition for funding rather than coordination of activities. Current 
reporting requirements for multifocal area projects increase operating costs, 
yet often do not capture synergies generated and trade-offs. 


Factors influencing program outcomes 


Programmatic approaches have been part of operations since the GEF’s 
establishment and were formally introduced in May 2008. Initially, pro- 
grams were de facto projects funded through phases with subsequent financ- 
ing tranches. Clustered programs, introduced after May 2008, included a 
set of “child projects” designed to contribute to the overall objective of the 
parent program. In 2015, the GEF introduced the IAP Programs, which 
focus on the main drivers of environmental degradation and support broad 
coalitions of committed stakeholders and innovative, scalable activities. 
Over time, programs have evolved from a narrow approach, largely focused 
on mitigating the negative effects of food and energy production on bio- 
diversity loss, land degradation, and climate change, toward applying an 
integrated approach encompassing a wider set of drivers such as food and 
energy production and consumption, buildings and infrastructure, con- 
struction, and transportation. 

The GEF portfolio of programs is sizable, diverse, and growing, account- 
ing for 8.7 percent of the total GEF funding as of April 2016. Compared 
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to stand-alone projects, programmatic projects performed better, but pro- 
gram complexity matters for outcomes. Child projects in programs gen- 
erally performed better than stand-alone projects on all rating dimensions, 
especially on execution quality, sustainability, and M&E design. However, 
performance declined with increased complexity. Although complex pro- 
grams are designed with a long-term perspective, they are substantially more 
difficult to execute than simple ones. The India Coastal and Marine Program 
is an example of a single-country, single-agency, single-focal area program 
composed of two homogeneous child projects, which successfully demon- 
strated multisectoral approaches to mainstreaming biodiversity conservation 
into economic activities in two marine ecoregions of the country. Both child 
projects have informed national policy actions. 

In comparison, the multifocal, multicountry Middle East and North Africa 
Desert Ecosystems and Livelihoods Program (MENA-DELP) was a nonho- 
mogeneous collection of individual national projects loosely related to each 
other through a regional “glue” project; these did not realize any additional 
benefits from their participation in MENA-DELP. We found that the aggre- 
gate outcomes and potential impacts of MENA-DELP do not differ from the 
sum of those of its child projects, apart from some inadequately aggregated 
M&E information and limited sharing of lessons learned among child project 
stakeholders (GEF IEO 2018c). 

Designing projects for broader adoption, a central concept for program- 
matic approaches, has improved substantially, but only limited actions toward 
its implementation tend to occur. Our analysis of terminal evaluations in 
programmatic approaches found that 31 percent of child projects indicated 
intent to promote broader adoption at the design stage, but only 13 percent 
took concrete steps toward this and 6 percent actually implemented broader 
adoption actions. The most frequently observed forms of broader adoption 
are mainstreaming (see Box 5.4) and replication. Scaling up has had little or 
no broader adoption, despite its being a leading rationale of program design, 
or effect on market change. 

Single-country programs, with stronger ownership and better alignment 
with country priorities, perform better than multicountry programs. With 
the exception of programs addressing transboundary issues (notably inter- 
national waters), GEF programs have progressively shifted over time from a 
single-country to a multicountry focus. STAR funds are a substantial share 
of total program resources, regardless of geographic scope. In general, the 
smaller the country’s STAR funds, the higher the share of its total STAR 
allocation is given to a program. This helps countries maximize their invest- 
ments and returns in terms of global environmental benefits. 

Program/child project coherence in objectives has improved in recent pro- 
grams, as the design of both programs and their child projects pays greater 
attention to broader adoption and program objectives are better defined. This 
improved coherence is notable in the design of highly complex programs, 
under which projects more specifically address program outcomes. 


86 GEF interventions 


Box 5.4 


Demonstrating program impacts through mainstreaming 


The China Integrated Environmental Management (IEM) Drylands Program 
aimed to address desertification, deforestation, and biodiversity loss resulting from 
land degradation in China’s western dryland provinces. The national government 
at that time was looking for solutions to massive soil erosion that had led to fatal 
floods in the lower Yellow River. The program introduced IEM, which brings 
together different sectors to address multiple environmental and socioeconomic 
issues in an integrated manner. Through interventions demonstrating IEM, ben- 
eficiaries reported improvements in ecosystem protection and vegetation pro- 
ductivity. While achieving these environmental outcomes, local incomes also 
increased through higher value and more diverse crops. These positive results 
prompted local governments to mainstream IEM principles into provincial, state, 
village, and township planning systems. Planning approaches in four out of six 
provinces have shifted from a top-down to a multisector integrated approach. 
Recommended IEM actions served as inputs to 26 county development plans, 
which have been incorporated into provincial and national budgets. 


(Source: IFAD 2016) 


Cost-eftectiveness and efficiency decline as programs become more multi- 
dimensional. We analyze cost-effectiveness in the GEF through three factors: 


1 Program and child project approval times as per the GEF project cycle 

2 Program financing and cofinancing 

3 Program effectiveness and efficiency, expressed by terminal evaluation 
ratings 


With this approach, if the costs of a program are less than the alternative (stand- 
alone project/cluster of projects), then the program is still more cost-effective. 
Overall, project cycle analysis shows no major differences between program- 
matic and stand-alone projects. Our analysis of terminal evaluations found 
that child projects scored higher on effectiveness and efficiency, and leveraged 
higher cofinancing than stand-alone projects (at 1:10 vs. 1:7), but efficiency 
ratings declined with increased program complexity. GEF Agencies’ diversity 
in terms of mandates and operational approaches makes collaboration chal- 
lenging. Program coordination is an added cost that increases with complexity. 

M&E has improved in the design of recent programs but still faces chal- 
lenges. Child projects achieved higher ratings for M&E design compared to 
stand-alone projects but show weaker implementation of M&E than their 
stand-alone counterparts. We have found that M&E ratings for projects in 
highly complex programs decline from design to implementation stages. 
Program M&E evidence would demonstrate the value addition of a program 
over a set of projects, but its availability is limited. Box 5.5 illustrates how 
design affects program outcomes and results. 
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Box 5.5 


The GEF Global Wildlife Program: lessons from design 


Of most direct relevance to the COVID-19 pandemic is the GEF’s program to 
combat illegal wildlife trade. The supply chain of wildlife and wildlife products 
from source areas, and transportation through local, national, and international 
networks to distant markets, allows for natural spillover and spread of zoonotic 
disease. Unsafe handling and other practices allow pathogens to pass from wildlife 
to humans. These conditions and practices are even more challenging to monitor 
—and impossible to regulate — in illegal wildlife trade. Illegal wildlife trade gen- 
erates $26 billion per year, making it the fourth most profitable illegal industry in 
the world. Stopping illegal trade has been a difficult endeavor, and unfortunately, 
as long as there is a demand for wildlife products, there will be trade. 

Several GEF-funded projects include activities related to combating illegal 
wildlife trade and address the drivers of biodiversity loss. 

Launched in 2015 and designed to be implemented over seven years, the Global 
Wildlife Program takes on the issue of illegal wildlife trade in a coordinated and 
comprehensive way. It involves 20 child projects in Asia and Africa, including 
one global coordination and knowledge management grant. The program takes 
a three-pronged approach to address illegal wildlife trade: stop poaching, stop 
trafficking, and reduce demand. Its main interventions have included supporting 
protected area management, reducing poaching of target species, engaging com- 
munities in managing human-wildlife conflict, improving performance across the 
enforcement and criminal justice chain, establishing partnerships, and knowledge 
management. Its success requires collaboration with other partners and interna- 
tional organizations, some of which are not GEF Agencies. 

The Global Wildlife Program’s comprehensive theory of change addresses the 
source of wildlife traded illegally, its shipment and transportation, and the mar- 
ket demand, with this supply chain unfolding in source range countries, transit 
countries, and destination countries. The theory of change also encompasses both 
short-term and long-term interventions to address illegal trade and ensure that 
wildlife resources are sustainably used. 

The IEO conducted a formative assessment of the program in 2017 (GEF IEO 
2018b). At that point, the program was in its early stages of implementation, and the 
evaluation provided insights into what works, why and under what circumstances, 
for whom, and the extent of the benefits. The lessons learned from the formative 
assessment allowed for timely improvements in program implementation. 

Our program evaluation found gaps in geographic and species coverage. 
For example, the program includes no countries from the Latin America and 
Caribbean region, even though substantial illegal wildlife trade occurs there. This 
is because the program emerged from a focus on the plight of charismatic meg- 
afauna in other regions, specifically the trafficking of elephant ivory, rhinoceros 
horn, and large cats. 

Most Global Wildlife Program funding is aimed at fighting illegal wildlife 
trade at the source through bolstering support for protected areas, supporting law 
enforcement efforts, creating opportunities for local communities to benefit from 
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conserved wildlife (rather than poached or traded wildlife), and development of 
alternatives and sustainable livelihoods for local communities. Minimal funding 
has been allocated to demand reduction and to transit countries, and this focus on 
source countries risks ignoring trafficking in species that originate outside a coun- 
try. The global nature of illegal wildlife trade requires addressing cross-bound- 
ary issues, in which transportation and logistics sectors — shipping lines, airlines, 
freight forwarders, and express couriers — all play a critical role. 

In another component of program design, the Global Wildlife Program only 
requires reporting of indicator data on arrests, prosecutions, and convictions as 
they are relevant to the individual project. Requiring reporting on all these com- 
ponents of the criminal justice system would enhance GEF efforts to combat cor- 
ruption and build political will. 

Moreover, the evaluation underscored the need for a comprehensive approach 
along the entire supply chain — from sources to global markets — to contain the 


spread of illegal trade. 


Integrated Approach Pilots: innovations and challenges 


The GEF introduced three programs in 2015 to achieve systemic change at 
scale by addressing the major drivers of global environmental degradation in a 
holistic way (GEF 2014b). These pilot programs, focused on cities, food secu- 
rity, and commodities, will test the delivery of more integrated approaches 
that address discrete, time-bound, global environment challenges. The IAPs 
incorporate multiple GEF and non-GEF Agencies, countries, and interven- 
tions designed to be integrated across focal areas. This section presents lessons 
from the formative review of the three pilots, and highlights key good practices 
and areas for improvement from our analysis of this pilot experience to date. 

The three IAPs are designed to build on existing linkages and connections 
across focal areas to address global environmental issues more holistically. 
The IAPs aim to support activities in recipient countries that help generate 
benefits that correspond to more than one global environmental objective by 
tackling the underlying drivers of environmental degradation (GEF 2020). 

The Sustainable Cities IAP Program (the Cities IAP) recognizes the chal- 
lenges and opportunities of rapid urbanization in developing countries. The 
program initially engages 23 cities in 11 countries to promote the integration 
of environmental sustainability in urban planning and management. 

The Sustainability and Resilience for Food Security in Sub-Saharan Africa 
IAP Program (the Food Security IAP) aims to support countries in safe- 
guarding and maintaining ecosystem services by investing to improve small- 
holder agriculture and food value chains. The program targets 10 million 
hectares of production landscapes with 2—3 million beneficiary households in 
drylands ecosystems of 12 Sub-Saharan African countries. 

The Taking Deforestation out of Commodity Supply Chains IAP Program 
(the Commodities IAP) has been designed through a supply chain lens 
for each of the three commodities responsible for 70 percent of tropical 
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deforestation globally — soy, palm oil, and beef (Brack et al. 2016). It will 
support activities in the four producing countries of Brazil, Paraguay, Liberia, 
and Indonesia, and in demand markets. This IAP aims to engage particularly 
with the private sector. 

In these three priority themes, the GEF has an important convening role to 
support and mainstream environmental management and perspectives. The 
hub projects can provide a regional forum for participating agencies, coun- 
tries, and others to discuss coordination, common strategy development, spe- 
cific technical and institutional assistance to countries, and a strategic learning 
agenda. The GEF is also vital in bringing together cofinancing resources. 

The three IAPs and related child projects have coherent designs in terms 
of alignment of program and child project objectives, results-based manage- 
ment frameworks, and M&E systems. However, alignment between project/ 
program results frameworks and tracking tools in terms of outcomes and 
indicators is lacking and uneven across the three IAPs. 


Designed for innovation 


Innovative features of the IAPs begin with their theories of change. The 
Commodities [AP has incorporated elements in its design and theory of 
change that address the major causes of deforestation in a holistic fashion 
by integrating the entire supply chain. The Food Security IAP theory of 
change integrates local and landscape natural resource management practices 
by promoting partnering and enabling policies (Engage), scaling-up of best 
integrated natural resource management practices (Act), and common meas- 
urements and learning (Track). This model has enabled a strong coherence in 
the formal design and multiscale approach of the child projects. At the child 
projects level, innovation in the Food Security IAP includes scaling up of 
new technologies and best practices in new areas, and broadening perspec- 
tives to a landscape approach. The Cities IAP is one of many interventions 
for urban sustainability, but aims to link with as many relevant initiatives as 
possible. An important innovation for the GEF is direct work with subna- 
tional governments for the implementation of Cities IAP child projects. Both 
the Cities and Food Securities IAPs bridge gaps between the ministries of 
environment and other government entities in the countries involved. 

The introduction of specific knowledge platforms and networks for 
cross-learning among child projects is a new approach for the GEF and a 
primary pilot feature in the three IAP Programs. National/global platforms 
and partnerships will need a strong evidence base to assess whether they 
can provide the support and momentum necessary to influence activities, 
perceptions, and sustainability. For the Cities IAP hub project, needed work 
includes creating a common framework across the child projects, develop- 
ment of a baseline set of indicators, and defining the hub’s role in capacity 
building. The Commodities IAP is a knowledge generation program primar- 
ily for the stakeholders involved at the global and national levels. The Food 
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Security IAP design supports knowledge exchange and enables the engage/ 
partnership component of the theory of change. 

The design of all three IAPs emphasize broader adoption, which is a pri- 
mary reason countries participate in the IAPs. We found that all of the IAP 
child projects include plans for sustaining project interventions and almost all 
outline specific measures for planned broader adoption of outcomes, replica- 
tion, and scaling up. 

The IAPs have addressed cross-cutting issues in their designs. Gender has 
been considered in most child projects and more than half of the projects 
have a gender mainstreaming strategy or plan in place, with specific targets 
for women’s participation in the child projects as beneficiaries and inclusion 
of gender-specific indicators. Resilience considerations are embedded in the 
IAPs and resilience is an integrating concept in almost all IAP child projects. 

Although most of the IAP projects show well-designed M&E systems, 
alignment between program/project results frameworks and tracking tools 
is needed in terms of outcomes and indicators. Inconsistencies in the expres- 
sion and measurement of global environmental benefit targets risk hampering 
program-level M&E. Data on these targets is scattered throughout program 
and project documents, and calculation methods vary. 


Complexity and challenges 


Designing and launching such complex programs raises challenges, including 
time required for approval and engagement with a wide set of stakeholders at 
the design stage. The three IAPs are complex programs — they are multifocal 
area and multicountry endeavors that engage with multiple GEF Agencies. 
Moreover, they seek to work at local, landscape, national, and regional lev- 
els, which adds considerable challenges to their implementation. Although 
complex programs may have better longer term sustainability and better 
M&E design, they are substantially more difficult to execute than are simple 
ones (GEF IEO 2017b, 13). Absence of clear criteria in country selection and 
agency selection based on comparative advantage for program implementa- 
tion, limited success in harnessing private-sector financing, and a wider set of 
stakeholders make the processes cumbersome and challenging. 


Notes 


1. Performance is measured on a six-point scale: 1 = highly satisfactory, 2 = moder- 
ately satisfactory, 3 = satisfactory, 4 = unsatisfactory, 5 = moderately unsatisfactory, 
and 6 = highly unsatisfactory (GEF 2017b). 

2. The North African countries are Algeria, Egypt, Libya, Morocco, and Tunisia; 
the East African countries are Comoros, Djibouti, Eritrea, Ethiopia, Kenya, Mad- 
agascar, Mauritius, Rwanda, Seychelles, Somalia, South Sudan, Sudan, Tanzania, 
and Uganda; the West Sub-Saharan countries are Benin, Cote d'Ivoire, Ghana, 
Guinea, Liberia, Nigeria, Sierra Leone, and Togo. These subregions correspond 
to a GEF constituency of member countries and are each represented in the GEF 
Council. 
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. Cofinancing Policy (GEF/C.46/09), GEF Council Documents, May 2014. 

4. This includes three GEF constituencies that consist of following countries: Benin, 
Cote d'Ivoire, Ghana, Guinea, Liberia, Nigeria, Sierra Leone, and Togo; Burundi, 
Cameroon, Central African Republic, Congo, Congo DR, Equatorial Guinea, 
Gabon, Sao Tome and Principe; and Burkina Faso, Cabo Verde, Chad, Guinea- 
Bissau, Mali, Mauritania, Niger, Senegal, and The Gambia. 

5. The UNEP Annual Performance Report 2018 (UNEP 2018) has a detailed analy- 
sis on project sustainability. 

6. World Wildlife Fund priority areas, Conservation International biodiversity hot- 
spots, Alliance for Zero Extinction sites, Important Bird Areas, Ramsar sites, or 
UNESCO World Heritage Sites. 

7. The definition of land degradation neutrality was approved by COP12 (July 2015). 
8. Complexity is a function of the degree of homogeneity of a program’s child pro- 
jects and whether they belong to one or multiple countries, agencies, and/or focal 
areas. Tested for statistical significance, the relation between the outcome ratings 
from available terminal evaluations and the four complexity factors has shown that 
complexity is a good predictor of outcomes and is inversely related to the out- 
comes: the higher the complexity, the lower the outcomes (GEF IEO 2017b). 
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6 Toward transformational 
change 


Incremental environmental strategies alone will not suffice given the unprece- 
dented nature of the pressures faced by the earth’s ecosystems over the coming 
decade. As stated in the 2020 vision statement of the Global Environment 
Facility (GEF), these pressures “compel the GEF to equip itself to promote 
transformational change and to achieve global environmental benefits on a 
larger scale” (GEF 2012). One way the GEF specifically aimed to achieve this 
vision was by supporting innovative activities that “are scalable across multiple 
countries, regions, and sectors through policy, market, or behavioral trans- 
formations.” However, when the statement was developed in 2012, the GEF 
had only a limited framework or evidence to understand the mechanisms and 
factors influencing such transformative change and the process of scaling up. 
The GEPF’s additionality — the outcomes that can be truly attributed to a GEF 
project or program — had been expressed and measured through the concept 
of incremental cost reasoning, which does not address the underlying mecha- 
nisms of transformative change or scaling up. To fill this knowledge gap, the 
GEF Independent Evaluation Office (IEO) developed an enhanced framework 
for understanding the GEF’s additionality, going beyond the incremental cost 
principle. Building on this improved concept of additionality, the IEO explored 
the underlying mechanisms that influence the outcomes to impact pathways, 
and contributing factors, through a review of GEF experiences in promoting 
transformational change and scaling up (GEF IEO 2018a). In fact, we find that 
the crucial mechanisms influencing the two processes are closely related; trans- 
formational chance and scaling up can be considered steps in a continuum, 
with scaling up as one possible outcome of transformational change. 


Beginning with additionality 


The GEF theory of change 


Figure 6.1 presents the GEF generic theory of change that responds to the focal area 
strategies to support the achievement of global environmental benefits (GEBs). At 
the same time, however, the theory of change has been an under-documented 
aspect of the GEF’s work, complicating assessment of the full additionality — or 
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Figure 6.1 Generic theory of change for the GEF 
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transformational change — that GEF interventions provide. GEF additionality 
also hinges on the link between the direct areas of GEF contribution and the 
environmental impact, raising the question of how the GEF supports broader 
adoption that leads to a positive cycle and strengthens environmental benefits. 

Drawing on recent thinking on additionality and the GEF theory of 
change, the IEO developed an enhanced framework for assessing the GEF’s 
additionality that goes beyond the concept of incremental cost reasoning and 
provides a basis for understanding the processes of transformational change 
and scaling up. As stated in Chapter 4, we define additionality as: 


e Changes in the attainment of direct project outcomes at project comple- 
tion that can be attributed to GEF interventions; these can be reflected in 
an acceleration of the adoption of reforms, the enhancement of outcomes, 
or the reduction of risks and greater viability of project interventions 

e  Spill-over effects beyond project outcomes that may result from systemic 
reforms, capacity development, and socioeconomic changes 

e Clearly articulated pathways to broaden the impact beyond project com- 
pletion that can be associated with GEF interventions 


The GEF expects that specific areas of contribution — achieved through GEF- 
funded projects — will have a catalytic effect that leads to broader adoption 
of successful interventions. This catalytic effect, and its broader adoption and 
behavioral change, then generates a virtuous cycle of environmental impact. 
Progress in GEBs is assumed to foster further behavioral change and expan- 
sion of sound environmental practices. 

Full additionality encompasses both a project’s outcomes and a project’s impact. 
Although not all projects can be linked to impact on GEBs that extends beyond 
the project outcomes, we can use this framework to distinguish projects with 
potential for a high impact, scaling up, or transformational change. Outcome 
additionality is directly attributable to project interventions. In contrast, impact 
additionality is the establishing of a plausible pathway for the project’s contri- 
bution to GEBs, recognizing that many of the benefits may only materialize 
following the completion of the project. Additionality typically results either 
from focusing entirely on GEBs, with no other funding sources available, or by 
enhancing viability, speeding up, and greening (see an example in Box 6.1). 


Creating transformational change 


Engagements that help achieve deep, systemic, and sustainable change with 
large-scale impact are regarded as transformational. Four specific criteria dif- 
ferentiate transformational interventions from those that are “merely” highly 
successful, complex, or large in size: 


1 relevance 

2 depth of change 
3 scale of change 
4 sustainability 
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Box 6.1 


Enhancing viability and speeding up 


The project Reducing Transboundary Degradation in the Kura-Aras Basin, a 
regional project in Armenia, Azerbaijan, and Georgia, aimed to create an ena- 
bling framework for the long-term, sustainable, integrated management of the 
river basin. This framework would follow integrated water resources management 
principles and prevent overuse and conflicting uses of water resources. 

The GEF project provided a regional framework and help in aligning plan- 
ning procedures at the national level, which supported development of a Kura 
Aras Environmental Program and formulation of a strategic action program and 
national integrated water resources management plans. Without the GEF project 
and its support, delivery of key global environment benefits, such as improved 
hydrological flows and reduction of persistent toxic substances, would have been 
delayed and perhaps even lost to the detriment of both the river basin and the 
Caspian Sea. 


By strategically identifying and selecting projects that address environmental 
challenges of global concern and are specifically designed to support funda- 
mental changes in key systems or markets, the GEF engages in interventions 
that are more likely to have a sustainable, large-scale impact. However, project 
impact assumes supportive contextual conditions and good implementation. 

To assess specifics of whether and how GEF projects achieve transfor- 
mational change, the IEO undertook an evaluation using a purposive sam- 
ple selected from 156 completed projects nominated by GEF Agencies! as 
having achieved transformational change, verified through independent, 
project-level evaluations (GEF IEO 2017). We developed a framework for 
understanding the process and the factors influencing transformative change, 
presented in the underlying theory of change in Figure 6.2. We screened the 
projects against this framework and also reviewed their characteristics against 
the established additionality parameters (see Chapter 4, Box 4.2). For the 
evaluation, we selected eight illustrative projects, taking into account their 
diversity in focus, regional distribution, and agency: 


Lighting Africa 

China Renewable Energy Scale-Up Program, Phase I (CRESP-I) 
Uruguay Wind Energy Program (UWEP) 

Sanjiang Plain Wetlands Protection Project (China) 

Sustainable Land, Water, and Biodiversity Conservation and Management 
for Improved Livelihoods in Uttarakhand Wind Sector Project (SLEM; 
India) 

6 Strengthening the Protected Areas Systems (PAS) in Namibia 

7 Amazon Regional Protected Areas Program, Phase I (ARPA-I; Brazil) 


AUN- 
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Figure 6.2 Theory of change for GEF transformational interventions 


8 Promoting Payments for Environmental Services and Related Sustainable 
Financing Schemes in the Danube Basin 


The evaluation used a cross-case analysis in combination with a meta-analysis 
of the project-level evaluations for each of the projects to assess the condi- 
tions (and combinations of conditions) that support transformational change. 
Based on the analysis, the IEO was able to identify four factors common to 
these projects that provide lessons for the future. 


Design with ambition 


First, these interventions that achieved transformational change had clear 
ambition at design. Their objectives aimed at profound, fundamental change 
in addressing a market distortion or a systemic bottleneck that was a root 
cause for a global environmental concern. Four of the cases focused primarily 
on system-wide transformation, taking a comprehensive approach to modi- 
fying the functioning of a collection of components (economy, public sector, 
private sector, community) whose interactions have environmental conse- 
quences. In the four other cases, the primary thrust was on transforming a 
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market (i.e., the supply and demand of goods and services) associated with 
environmental impacts of global concern. 


Address policies for market and system changes 


Second, the adequacy of the policy environment had an important impact on 
the depth and scale of reforms promoted by the transformational interven- 
tions. All of the projects, thus, addressed market and system changes through 
policies. In three cases, the interventions had a major role in helping define 
and implement the main policies essential to trigger and sustain transforma- 
tional change. In China and Uruguay, the projects had a strong influence 
on policies that effectively stimulated development of renewable energy. In 
Namibia, the project provided technical support for the drafting of new pol- 
icies for the Ministry of Environment and Tourism, affecting the protected 
area systems. In three other cases, the interventions played a modest role 
in strengthening the policy framework needed to support transformational 
change. These ranged from discussing with the governments of Ghana and 
Kenya lowering impact taxes as an enabling environment for the solar lamps 
market, to proposing prohibition of animal grazing and fishing in all nature 
reserves in the Sanjiang Plain in China, to mainstreaming payment for envi- 
ronmental services concepts into national fisheries policies in Bulgaria and 
Romania. In India, the state government of Uttarakhand granted the local 
rural government formal legal recognition for watershed development plan- 
ning. In Brazil, ARPA-I used the existing legal context for protected areas 
to involve many government agencies and financing partners to demonstrate 
the practicality of a participatory approach to the establishment and manage- 
ment of protected areas. 


Build for financial stability 


Third, these transformational interventions established a mechanism for 
financial sustainability by integrating within government budgetary sys- 
tems or by leveraging market forces and key stakeholders’ economic inter- 
ests. In China, CRESP-I supported a feed-in tariff for renewable energies 
that provided financial returns attractive enough to encourage state-owned 
and private enterprises to accelerate their investment in renewable energy. 
In Uruguay, wind power investment licenses were allocated through a fair 
bidding process that guaranteed access to the grid. The resulting prices were 
competitive with those of fossil fuel alternatives and have gradually declined 
further as a result of growing efficiencies and technological improvements. 
In Uttarakhand, project beneficiaries have an incentive to maintain water 
harvesting structures, namely, their own investment through cost sharing. In 
the Sanjiang Plain, a portion of local county revenues generated from forest 
development activities is used to meet the financing requirement for nature 
reserve management. 
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In four cases, the funds provided by the GEF were supplemented with 
important financial contributions from international donor partners, which 
enabled the projects to expand their scope and scale. In the remaining inter- 
ventions, the GEF’s support was supplemented by funding from the imple- 
menting agencies and the governments and in one case by counterpart 
funding from the government alone. 


Focus on implementation quality 


The fourth common feature was that all interventions were well implemented 
in terms of quality of project design, supervision by the GEF Agency, and 
effectiveness of the executing agencies. Some of the salient features that drove 
implementation quality included comprehensive diagnostic assessments to 
identify barriers; coherent designs to target all identified barriers; and early 
involvement of strong executing agencies that would own the project objec- 
tives and be willing to learn, adjust, and adapt the design, scope, and man- 
agement as needed to ensure success. 

Several other factors contributed to achieving transformational change. 
These included government ownership of and support for the project, imple- 
mentation capacity of local institutions (other than the main executing 
agency), civil society and local community participation, private-sector par- 
ticipation, and economic and market conditions. 

Six of the sample cases demonstrated strong government ownership that 
contributed significantly to the projects’ satisfactory outcomes. The Lighting 
Africa project was specifically designed to catalyze a private-sector-driven 
sustainable market transformation. It was not country-specific and did not 
involve the governments except to discuss policy changes, such as the low- 
ering of import taxes. However, even with relatively limited public policy 
dimensions, securing buy-in from local governments can greatly reduce risks 
of a national government setting adverse expectations and incentives. Thus, 
the program’s success in Kenya was facilitated by some support from the gov- 
ernment, while in Ghana, the government’s focus on grid extension prom- 
ises and relatively dismissive attitude toward portable off-grid solar solutions 
likely dampened private-sector interest in the market. 

The implementation capacity of local institutions can play a major role in 
project outcomes, especially when the activities are spread over a range of 
sites and local jurisdictions. The CRESP, Sanjiang Wetlands, Namibia PAS, 
and Uttarakhand SLEM projects included targeted activities to strengthen 
the local institutional capacity, all of which were effective in contributing to 
the projects’ success. 

The Namibia PAS projects engaged local civil society and community 
organizations in key contributing roles by supporting engagement between 
park personnel and neighboring communities. Animportant catalyst for coop- 
eration between the two groups was funding for game translocations from 
protected areas to conservancies. The communal and private conservancies’ 
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rights to use and benefit from wildlife on their land gave them a direct inter- 
est in cooperating with the protected areas that supply their wildlife. 

Uttarakhand SLEM had a high level of community participation in its 
various components, which increased the likelihood that the activities would 
be continued after project completion. Fifty-five partner agencies supported 
outreach to build this community connection, including NGOs, academic 
institutions, and the private sector. They provided overall project implemen- 
tation support, social mobilization, participatory monitoring and evaluation 
(M&E), and technical assistance. 

The impact of private enterprises on the effectiveness of the transforma- 
tional interventions was mainly defined by the extent of their (supply-side) 
response to the changes the project created. As expected, the response was 
strongest where market change was at the center of the interventions. Thus, 
CRESP and UWEP contributed to the successful transformation of the wind 
energy market in China and Uruguay, respectively, by addressing the barriers 
that had constrained the market’s development, most importantly by helping 
establish a feed-in power tariff that made it financially attractive for pri- 
vate investors to invest in wind energy. The Lighting Africa program helped 
catalyze the market by creating awareness and demand for quality, afforda- 
ble solar lamps while also stimulating the supply chain by providing market 
intelligence; developing a quality assurance infrastructure; helping govern- 
ment address policy barriers; and facilitating access to finance for manufac- 
turers, local distributors, and consumers. 

Interestingly, transformation can be achieved by projects of different size. 
While most of the projects included in the sample were multimillion-dollar 
efforts with long durations, the projects in Uruguay (see Box 6.2) and the 
Danube Basin were relatively small with a limited duration but targeted 
important barriers and worked with key stakeholders at the right time. The 
evaluation framework and findings can help the GEF — and potentially other 
organizations — gauge project concepts in advance to determine their prob- 
ability for supporting transformative change and enhance project designs. 


Transformational outcomes 


Our evaluation found that transformational outcomes were achieved through 
mainstreaming, replication, and catalytic effects. 

Mainstreaming refers to the integration of the practices, policies, and 
programs promoted by the project into those of the country or local juris- 
dictions, as appropriate. The successful mainstreaming of environmentally 
positive policies and programs is perhaps best illustrated by the ARPA-I pro- 
ject, which supported the creation and consolidation of protected areas and 
the establishment of an endowment fund to meet a portion of their oper- 
ational costs. Upon completion, the project had not only doubled the area 
of Brazilian Amazon under strict protection, but also proven to all major 
stakeholders — including federal and state governments, local peoples, NGOs, 
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Box 6.2 


Creating the wind power market in Uruguay 


In 2000, Uruguay’s power system depended fully on hydropower and imported 
fossil fuels. With the country’s hydropower potential practically exhausted, its 
growing electricity demand would require gas-fueled power plants, which not 
only rely on imported energy but transmit impacts of international gas price fluc- 
tuations onto the national economy and increase greenhouse gas emissions. Facing 
this situation, the government of Uruguay recognized the long-term potential in 
developing local energy resources — such as wind and biomass — and established 
the legal basis and framework for promoting them. Barriers to this development 
included: 


e Insufficient or inappropriate regulations for the installation and operation of 
wind farms, including grid access and dispatch 

e Lack of an enabling policy framework for power purchase agreements 
between wind power suppliers and the national power company” 

e Underdevelopment of technical standards, building codes, and environmen- 
tal guidelines for wind energy systems 

e Financially unattractive returns for private wind energy projects 

e Insufficient wind energy knowledge and capacity among both public- and 
private-sector actors 

e Lack of a mandate for the national power company to promote and deploy 
wind energy systems 

e Lack of financial resources and technical equipment to gather data on 
Uruguay’s wind resources 


In 2007, UWEP launched to establish a 5-MW demonstration project, with 
activities expressly aimed at removing each of the identified barriers. Specifically, 
UWEP supported the creation of an enabling policy framework for wind energy, 
including regulations for construction and operation of wind farms, access and 
dispatch to the network, technical codes, and financial incentives. It strengthened 
capacity and business skills to prepare and implement wind energy technology 
with public and private delivery models. It also addressed technological barriers 
by providing measuring equipment and implementing a pilot 5-MW wind power 
plant connected to the grid. Following UWEP’s completion in 2012, the final 
evaluation report (Rodriguez 2013) concluded: 


With the decisive participation of this project, an enabling legal and regula- 
tory framework was established for the development of wind energy in the 
country. A transparent market for wind power was created and 43.45 MW 
have been introduced in the country through December 2013, and several 
projects are in development which by December 2015 are expected to total 
990 MW, far exceeding project goals and converting wind power into a 
major energy source for the country. 
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This change in energy source directly avoided annual carbon emissions estimated 
at 0.86 million tons of CO, in 2015, an increase from zero tons avoided in 2007. 
Key factors in the project’s transformational success included: 


e The quality of the project’s design, which reflected a coherent, logical frame- 
work from the identification of barriers to the planning for their removal 
through specific activities, with appropriate institutional arrangements and 
implementation strategy 

e The timing of the project, which came at an unusual moment when the gov- 
ernment had made a strong commitment to renewable energy, as reflected 
in its establishment of an enabling legal and regulatory framework and its 
willingness to leverage the GEF medium-size project by cofinancing a major 
share of project costs 

e The creation of a competitive and transparent wind energy market with a 
stable framework for investments and adequate tariff incentives that elicited a 
strong private-sector response 

e The project’s inclusion and strengthening of a core of wind power specialists 
at the national power company, who helped with the preparation of technical 
standards and enabled the company to positively respond to the wind energy 


development mandate through both its own (public) and private investments 


private-sector organizations, and international donors — that effective pro- 
tected area creation and management could have a real impact in reduc- 
ing deforestation and protecting not only biodiversity but the rights of local 
peoples. Thus, the federal and state governments were fully committed to 
implementing the protected areas staffing and management plans instituted 
under the project, and donors and private enterprises continued to contribute 
to the endowment fund that covers most of the operating costs of these areas. 
Largely as a result, replication of the ARPA model continued in Phase II of 
the program. 

Demonstration and replication occur when the process or transmission 
channel established by the intervention continues to expand the outcome 
beyond the initial target area. Most of the projects achieved a substantial 
demonstration effect (i.e., initial impact): the practices and programs intro- 
duced by the project were adopted and replicated in similar contexts across 
an expanding geographical scope. For example, the integrated watershed 
management and conservation model introduced by the Sanjiang Wetlands 
project was adopted by six additional reserves beyond the initial six sup- 
ported by the project. In the Lighting Africa program, the overall approach 
that was initially piloted and successfully demonstrated in Kenya (albeit less 
successfully in Ghana) is currently being replicated in ten additional countries 
in Africa. 

Catalytic effects are aspects that extend beyond the intervention, such as 
synergies and complementarities among different instruments and interven- 
tions that lead to impacts greater than the sum of the interventions. The 
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most notable examples of catalytic effects involve the transformation of the 
market or system for renewable energy development. CRESP-I is credited 
with contributing substantially to the transformation of China’s renewable 
energy sector from an early piloting and demonstration stage to its status as a 
global leader in wind energy generation and the manufacture of wind power 
generation equipment. With an original target of 10 GW of installed wind 
power capacity, the project supported policy reforms, capacity building, and 
technology improvement that that resulted in an actual capacity increase to 
29.6 GW by 2010 and 129.3 GW by 2015. Similarly, UWEP has decisively 
supported the establishment of a legal and regulatory framework that cata- 
lyzed the creation of the wind energy market in Uruguay, which grew from 
virtually nothing in 2007 to 43.4 MW by the end of the project in 2013, and 
reached 990 MW in 2015 (ee Box 6.2). Finally, Lighting Africa catalyzed 
the creation of a commercial market for quality, affordable solar lighting in 
Africa; by 2014, more than 680,000 Lighting Africa-certified lamps had sold 
in Kenya, and almost 2 million lamps in other African countries. 

In this sample, the cases that produced catalytic effects were the three 
projects focused on renewable energy and climate change, and none of the 
others. These indicate a link between large-scale catalytic effects and tech- 
nological improvements with benefits that can be captured by harnessing an 
effective market demand. Thus, the decline in costs of renewable energy in 
relation to those of conventional fossil fuel electricity opened up new and 
economically feasible market opportunities that the interventions were able 
to exploit, with their attendant synergistic or catalytic effects. In other types 
of interventions — such as those focused on biodiversity protection and land 
conservation — the projects’ support for cutting-edge science and technolo- 
gies faced greater challenges in capturing and monetizing the related benefits. 
As a consequence, their transformational impacts appear linked mostly to 
establishing and mainstreaming institutional support mechanisms, with only 
partial reliance on market-based approaches. 


A closer look at scaling up transformational change 


Over 25 years, the GEF has made a clear shift from site-level pilot projects 
toward projects and programs implemented on larger scales. This is due in 
part to a better understanding of what interventions work based on the port- 
folio of demonstration projects implemented during the GEF’s early phases. 

The GEF 2020 Strategy (GEF 2015) particularly emphasized that large- 
scale impact can be achieved in three ways: through GEF interventions being 
scaled up by others, through market or behavioral transformation, and by the 
intervention working directly at a large scale. The strategy envisioned the 
GEF’s contribution to scaling up to be: 


* mainstreaming environmental priorities into broader policies, strategies, 
programs, and actions; 
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e working on supply chains and with industry-wide approaches; 
e implementing larger programs; and 
e cofinancing and leveraging via innovative financial instruments. 


The strategy also identified several enabling conditions for scaling up 
impact, including incentives, policies and regulations, institutions and 
institutional capacity, partnerships and coordination, financing, learning, 
and M&E. To operationalize this strategy, the GEF introduced the inte- 
grated approach pilots in GEF-6 and the impact programs in GEF-7, which 
began implementation recently and, thus, have only limited evidence on 
outcomes. 

To better understand the scaling-up process in depth and the influencing 
factors and conditions, the IEO developed a framework for scaling up build- 
ing on the understanding of GEF additionalities and the process of transfor- 
mational change. We also conducted quantitative and qualitative analyses on 
a purposive sample of successful and less successful cases of GEF support for 
scaling up. 

The GEF defines scaling up impacts increasing the magnitude of GEBs 
and/or expanding the geographical and sectoral areas where they are gen- 
erated to cover a defined ecological, economic, or governance unit. Scaling 
up is a continuous process that often takes place over longer time horizons; 
as such, scaling-up objectives need to be continually set and achieved until 
impacts are generated at the magnitude and scope of the targeted scale. 


Scaling up mechanisms 


The GEF contributes to the scaling-up process in two ways: 


1 Funding the implementation — including the piloting — of interventions 
that generate GEBs 

2 Supporting enabling conditions that allow these interventions to gener- 
ate impact at scale 


In addition to the replication and mainstreaming mechanisms mentioned 
in the previous discussion of transformational change, linking interventions 
is another channel through which impacts are scaled up. Linking refers to 
implementing different types of interventions across the multiple compo- 
nents (geographic locations, administrative levels, or sectors and institutions) 
of an ecological, economic, or governance system. For example, within value 
and supply chains, linking takes place between interventions that address 
causes and effects, such as working both in countries where deforestation or 
wildlife poaching occurs and in countries with high demand for forest and 
wildlife resources. Market change may be one form of linking within value 
chains when it addresses both supply and demand sides. Scaling up through 
creation of links can also involve implementing multiple interventions under 
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Figure 6.3 Process of scaling up 


a common theme or transboundary issue, such as water pollution or fisher- 
ies. Approaching scaling up through linking is consistent with the GEF’s 
shift toward addressing drivers rather than symptoms of environmental deg- 
radation to achieve more sustainable impacts at scale. With this focus, the 
programmatic approaches have increased GEF support to multistakeholder 
platforms at the regional and global levels. 

Although contextual factors affect the scaling-up process, GEF support can 
influence these factors in selecting partner individuals and institutions and by 
leveraging changes in the social-ecological context to align with scaling-up 
objectives. The GEF also influences contextual factors to be more favorable 
toward scaling up by supporting enabling conditions. Figure 6.3 shows which 
enabling conditions are most relevant for each component of the scaling-up 
process to influence the corresponding contextual factors. 

We analyzed 20 cases covering 38 projects in our evaluation of GEF sup- 
port to scaling up (GEF IEO 2019); see Table 6.1 (projects within these cases 
are included in Appendix 4). 

The cases analyzed cover a time span of 20 years from the pilot phase of the 
GEF to GEF-5; the earliest project, Mexico LUMEX, started implementa- 
tion in 1994. The most recent project in the study, the second phase of the 
Philippines CCA case, started in November 2014. 


Table 6.1 Cases in the evaluation of GEF support to scaling up impact, 2019 


Scaling-up case 


Focal area 


Scaling targets 


Target geographical scale 


Bangladesh 
IDCOL 

Brazil ARPA 

Brazil Rio Rural 

China CHUEE 

China CRESP 


China DDT 


China IEM 


China Termite 
Control 
Costa Rica PES 


Ethiopia SLM 


Indonesia 


COREMAP 


Climate Change 
Mitigation 

Biodiversity 

Land Degradation/ 
Multifocal 


Climate Change 
Mitigation 

Climate Change 
Mitigation 

Chemicals and 
Waste 


Land Degradation/ 
Multifocal 


Chemicals and 
Waste 
Biodiversity 


Land Degradation/ 
Multifocal 
Biodiversity 


Expand access to solar home systems in rural areas 


Expand and consolidate the protected areas system 
Demonstrate and increase adoption of biodiversity-friendly and climate- 
friendly agricultural practices through integrated ecosystem management 


Develop partnerships and capacities for a commercially sustainable delivery 
mechanism for energy efficiency projects 

Demonstrate and increase installed renewable energy capacity to reduce 
carbon emissions 

Dispose of DDT waste, eliminate production and consumption of 
dicofol, demonstrate and replicate integrated pest management 
technology, commercialize alternatives to DDT and TBT in antifouling 
paint 

Demonstrate and establish enabling conditions for adoption of integrated 
ecosystem management in agricultural areas adjacent to protected 
areas 

Eliminate production and consumption of chlordane and mirex by termite 
professionals 

Increase the area of forest under protection and sustainable management 
through payments for environmental services in private lands adjacent to 
protected areas 

Increase area of agricultural land under sustainable land management 


Demonstrate and establish framework for community-based coral reef 
management 


Country 

Amazon region 

Northwest region of 
the state of Rio de 
Janeiro 

Country 

Country 


Country 


Dryland ecosystems 


Country 
Country 
Vulnerable watershed 


areas in country 
Country 
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Scaling-up case 


Focal area 


Scaling targets 


Target geographical scale 


Macedonia PCB 
Mauritius POPs 
Mexico Ilumex 


Namibia 
NACOMA 


Philippines CCA 
Romania IW 
Senegal Ecovillages 
Uganda PA 


Uruguay Wind 
Energy 


Source: GEF IEO 2019 
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Build capacity for and promote adoption of weather-based insurance index 


Increase use of agricultural practices that reduce nutrient discharge to the 
Danube River and the Black Sea 

Demonstrate integrated approach to reduce carbon emissions, protect 
biodiversity, and create livelihood opportunities in rural areas 

Strengthen institutional capacity for long-term biodiversity conservation 

Demonstrate wind power plant and remove barriers to commercial 
investments in wind energy 
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In these cases, the GEF has provided scaling-up support in four distinct ways: 


1 Piloting and scaling up were planned for and implemented within the 
same project through different components. 

2 Piloting and scaling up were planned for at the design stage of the pilot 
project, and implemented through multiple consecutive or parallel 
projects. 

3 Piloting and scaling up were implemented through consecutive GEF 
projects based on results of the pilot stage. 

4 Piloting was supported by GEF projects, while the scaling-up stage was 
funded through other sources based on results of the pilot stage. 


In a majority of projects, scaling up was not planned and budgeted for at the 
outset, but was contingent on the success of the pilots. 


Scaled outcomes 


We found that scaling up was achieved in 95 percent of the cases analyzed 
through replicating interventions over a wider geographical area. At the same 
time, 16 of the 20 cases also aimed to mainstream the implementation of 
interventions within plans and programs at different levels of government, 
and in some cases across different government agencies. Only four cases 
used linking in addition to the two other modes to scale up impact. These 
four cases addressed specific environmental issues through multiple sectors, 
although linking was not planned from the beginning in all cases. This is not 
surprising and reflects earlier GEF project designs; only in 2014 did the GEF 
increase focus on scaling up through linking interventions across sectors. 

The extent of GEF support for scaling up and the rate at which outcomes 
are scaled up vary across focal areas but typically take place over more than 
five years. They generate higher outcomes per GEF dollar per year during the 
scaling-up stage than the pilot stage. Within cases where project documents 
specified GEF support for distinct piloting and scaling-up stages, measurable 
outcomes per dollar per year during the scaling-up stage were 1.1-74.5 times 
larger than during the pilot stage, indicating greater cost-effectiveness and 
higher cofinancing leveraged per GEF dollar. 

For example, in Senegal, the GEF supported the piloting of 10 Ecovillages. 
During the course of the project, initial successes allowed the national gov- 
ernment to replicate the approach in 84 more villages without additional GEF 
funding by reallocating GEF funds mainly toward training and livelihoods 
and by tapping civil society and private-sector funds for renewable energy 
infrastructure. By the end of the project, the national government had scaled 
up the approach at the national level by initiating replication in an additional 
400 villages through an agency created specifically for this purpose. 

The ARPA-I project, a transformative project, is credited with help- 
ing to double the area of Brazilian Amazon under strict protection — from 
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12 million ha in 2004 to more than 25 million ha in 2009. UWEP sup- 
ported the creation of the wind energy market in Uruguay, supplying about 
33 percent of the nation’s electricity needs in 2016, up from 0 percent in 
2008. The Namibia PAS projects improved the management effectiveness of 
98 percent of the country’s protected areas, while estimated populations of 
the lion, leopard, cheetah, and wild dog doubled from 2004 to 2012. 

In the case of interventions in payment for ecosystem services (PES) in 
Costa Rica, the first GEF project brought 130,900 ha of land under PES con- 
tracts in six years with $8 million in GEF grants. The second project placed 
another 166,004 ha of land under PES contracts in five and a half years with 
$10 million in GEF grants. The rate at which forests were protected under 
PES contracts was 11 percent higher during the scaling-up stage than the 
pilot. 

In the case of climate change adaptation in the Philippines, 607 farmers 
benefited over six and a half years with an initial $5 million GEF grant, 
while the scaling-up project covered 2,413 beneficiaries in three years of 
implementation with $1.1 million in GEF grants. The rate of beneficiaries 
covered during the scaling-up project was almost 40 times higher than that 
during the pilot project. Nearly half of the farmer beneficiaries (46 percent) 
were women. 


Factors influencing the scaling-up process 


The GEF has supported scaling up by establishing enabling conditions, 
choosing the appropriate influencers and institutions to work with, estab- 
lishing sustainable sources of financing, strengthening institutional capacities 
and gaining political support through participatory processes, and leveraging 
contextual conditions at the right time. GEF funding supports eight types of 
enabling conditions that contribute to the scaling-up process: 


e To motivate adoption of interventions: (1) knowledge and informa- 
tion dissemination, (2) participatory processes, and (3) incentives and 
disincentives 

¢ To allow sustained support for scaling: (4) institutional and individual 
capacities, (5) policy framework and operating guidelines, and (6) sus- 
tainable financing 

¢ To allow the scaling-up process to be adaptable and cost-effective in the 
face of changing contextual conditions: (7) multistakeholder interactions 
and partnerships and (8) systematic learning mechanisms 


Motivating adoption 


We found that governments often made scaling up a priority because the 
intervention was part of their existing development plans and policies or 
was a response to urgent external events. Other important drivers for scaling 
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up include commitments to the global environmental conventions, require- 
ments for European Union accession, national laws for which implementation 
is mandated, or even loans that need to be paid back to the World Bank and 
other financial institutions. In some cases, the political priority to scale up 
was further motivated by an external event such as a national crisis or inter- 
national pressure to expand interventions that would generate both envi- 
ronmental and social benefits. For example, in Bangladesh, a GDP growth 
of more than 6 percent created an increasing demand for electricity access. 
However, the electricity grid was growing slowly and often experienced sup- 
ply shortages. This made the promotion of solar home systems a priority for 
the government. High political priority can also sustain scaling-up activities. 
In Brazil, for example, Congress voted to undermine the ARPA program’s 
gains by degazetting federal protected areas in the Amazon in 2017. But 
pressure from national and international stakeholders, especially civil society, 
contributed to the president vetoing that decision in the same year. 

Participatory processes and evidence of benefits disseminated through 
knowledge-sharing activities can contribute to scaling up becoming a polit- 
ical priority. For example, in China’s Hai Basin, farmers in pilot counties 
earned more income from farming while reducing groundwater use as a 
result of GEF support introducing the use of remote sensing analysis to assess 
existing resources against water needs. Consequently, the government moved 
to scale up the intervention with additional GEF support. In contrast, despite 
a high level of engagement of farmers in India under the SLEM program, at 
least one project did not engage the district and state governments, leading to 
participatory land use plans not being incorporated into laws and guidelines. 
The program has likewise not achieved its objective of scaling up sustainable 
land management at the national level. 


Sustaining support 


In 75 percent of the cases analyzed, the GEF helped develop a policy frame- 
work or operating guidelines for adopting an intervention at scale. In climate 
change mitigation, this typically took the form of regulations for reducing 
private-sector costs to invest in new technology and setting standards for 
manufacturing the technology. In biodiversity and land degradation projects, 
GEF support for policy frameworks allowed the mainstreaming of more sus- 
tainable approaches into plans at national and local government levels. In 
the international waters focal area, the strategic action program approach 
allows national-level activities to contribute to regional-level impacts. This 
approach also uses formal endorsement to secure countries’ commitments to 
actions. A previous IEO study on the GEF’s support for legal and regulatory 
frameworks found that this type of support has contributed to scaling up 
interventions (GEF IEO 2018b). 

Advocates for implementation, at various administrative levels, ensure that 
efforts are sustained. In Senegal, for example, the main champion for the 
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Ecovillages initiative was the country’s president himself; in Costa Rica, hav- 
ing a technically competent champion in an influential position, such as the 
minister of environment, has proven to be an important driver for sustain- 
ing scaling-up support. That project also identified champions within GEF 
Agencies. In contrast, the SLEM program in India chose to work through 
state-level land use committees to develop land use plans, but these committees 
had no convening power. As a result, mainstreaming did not happen in other 
government agencies as planned. Another factor in gaining influencers’ support 
is the comparative advantages of the agencies that the GEF selects as partners. 

In all of the cases analyzed, GEF support contributed to building institu- 
tional and individual capacities for scaling up. This included establishing or 
strengthening government agencies that took on key roles for implementing 
an intervention at scale. In ARPA-I in Brazil, for example, very early GEF 
support helped establish FUNBIO, the organization that is now implement- 
ing GEF-supported projects on the ground. At the individual level, capacity 
building has included training stakeholders on how to implement an inter- 
vention, such as villagers in the use of solar panels for electricity, or staff of 
mining companies on careful handling of transformers with PCBs. 

The use of existing structures and mechanisms for implementing an inter- 
vention are important for sustainable scaling up. Examples are the network 
of termite control stations in China to help eliminate chlordane use, and the 
microfinance institutions already active in rural Bangladesh to promote solar 
home systems. Such structures and mechanisms typically have a long-term 
presence and wide geographic coverage. They, therefore, have the capacity 
and experience to implement and follow-up on interventions over a large 
area beyond a project or program’s duration. This approach is also more 
cost-effective. For example, when implementing Costa Rica’s PES program, 
the government used its protected area field offices to house the forest engi- 
neers who would be reviewing applications from landowners. In Bangladesh, 
the existing network of microfinance institutions was a decisive factor in 
expanding solar home systems in rural areas. 

Partnering with supporting institutions that have a long-term outlook is 
important for sustaining the momentum of the scaling-up process beyond 
one project. In most of the cases we analyzed, resources of other stake- 
holders supported scaling-up initiatives. These included not only bilateral 
donors but also civil society organizations and private companies. ARPA 
and COREMAP are examples of how donors’ long-term outlook from 
the beginning led to support that has helped sustain scaling-up initiatives 
through severe political and economic crises, such as a presidential impeach- 
ment and budget freeze in Brazil and the Asian financial crisis in Indonesia. 
With GLOBALLAST, the GEPF’s sustained strategic vision to eliminate inva- 
sive alien species by managing ships’ ballast water, successive replenishment 
phases supported project efforts over almost 20 years amidst extended negoti- 
ation processes among governments and shipping companies. Having a long- 
term outlook also allows partners to adapt how a project is implemented to 
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keep it consistent with the long-term scaling-up objective despite temporary 
setbacks and unfavorable contextual conditions. 

Sustainable financing of scaling-up efforts allows replication to continue 
and maintains other enabling conditions. Some GEF contributions have taken 
the form of market-based mechanisms, trust funds, or a mainstreamed gov- 
ernment budget allocation; in other cases, sustainable financing was provided 
through government initiatives, donor projects, or other international donors. 
When a national government takes ownership of an intervention by making 
it a priority, it invests significant long-term funding. This investment signals a 
degree of stability that in turn attracts funding from other donors and the pri- 
vate sector. For example, under UNIDO’s Program for Country Partnership, 
Ethiopia has invested $900 million over four to five years for infrastructure 
projects and is actively pushing the program’s agenda. This has led to long- 
term partnerships with the European Investment Bank, Food and Agriculture 
Organization (FAO), Italy, Switzerland, and others. At the regional level, one 
recent example of a GEF-supported sustainable financing mechanism is the 
private-sector partnership hub, which will allow integrated coastal manage- 
ment to be further scaled up in East Asian seas through the Partnerships in 
Environmental Management for the Seas of East Asia (PEMSEA).? 

In the absence of sufficient financing, interventions planned with an explicit 
long-term outlook can fail. In Senegal, a GEF-supported project that intro- 
duced sustainable-use community nature reserves as part of the country’s pro- 
tected area system was designed to be implemented in three phases. The first 
two phases successfully established 26 community nature reserves that linked 
fragmented ecosystems across 270 villages and created a network of mutual 
savings groups that provided financing to local entrepreneurs. However, fund- 
ing for the third phase fell through. Despite the project’s initial socioeconomic 
benefits, many of the alternative livelihood activities were discontinued due 
to a lack of funds to purchase equipment that would allow community mem- 
bers to apply the training they received in the first two phases. Operations at 
the community reserves continue at present, but at a minimal scale. 

Long-term sustainability of scaling-up activities is vulnerable to political 
and economic changes. An octopus fishing ban in Mauritius that was scaled 
up from the outer island of Rodrigues to the national level is now funded by 
the national government. The GEF and other donors invested in awareness 
campaigns and community training programs through multiple consecutive 
Small Grants Programme projects. However, the outcomes so far have been 
much lower than in the pilot for two reasons; first, the larger area needs a 
higher investment in enforcement efforts, and second, legislation at national 
level did not apply the ban to the entire supply chain, as was done in the pilot. 


Staying flexible and cost-effective 


Multistakeholder interactions and partnerships are important in coordi- 
nating multiple mandates, objectives, and activities among stakeholders, 
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which in turn is essential in keeping implementation at scale cost-eftec- 
tive. For example, the PEMSEA series of projects, supported by the GEF’s 
international waters focal area through the United Nations Development 
Programme (UNDP) for more than 25 years, has scaled up integrated 
coastal management from a few pilot sites to a region-wide interven- 
tion largely through multistakeholder partnerships. Among the PEMSEA 
activities are: 


e Regional networks for local governments, research institutions, and legal 
experts 

e Regional governance structures such as a high-level forum composed of 
environmental ministers of participating countries in the region and a 
partnership council with representatives from the national and local gov- 
ernments, communities, NGOs, research and educational institutions, 
the private sector, and regional and international organizations 

e The triennial East Asian Seas Congress that allows regional stakeholders 
to have dialogues, share lessons, and formally endorse regional targets to 
which each of them contributes 


Learning from previous projects can shape the design of subsequent pro- 
jects. Systematic learning mechanisms in GEF projects are usually in the 
form of knowledge exchange networks and regular multistakeholder meet- 
ings. Por example, the RERED project in Bangladesh integrated system- 
atic learning in its design to scale up successful models adaptively. While 
incorporating lessons from previous experience in Bangladesh and other 
countries, the project design included a provision to scale up support for the 
model with the most promise. Throughout its implementation, the project 
continuously incorporated lessons from its own pilot approaches, and as the 
national demand for the solar home systems grew, the project shifted its 
focus to that component. Ultimately, the project scaled up support to the 
most successful model, which used microfinance ownership rather than a 
fee-for-service approach. Within this model, the project also used M&E 
data from the field to incorporate new specifications and technologies (such 
as LEDs) in solar home systems to better serve lower income households, 
which in turn made the systems more attractive to a larger population 
(World Bank Group, IEG 2014). 


Notes 


1. The World Bank, UNDP, UN Environment, the FAO of the UN, and the Asian 
Development Bank. 

2. In this case, the national power company refers to the National Administration of 
Power Plants and Electrical Transmissions (Administración Nacional de Usinas y 
Transmisiones Eléctricas). 

3. www.pemsea.org/. 


114 Toward transformational change 


References 


GEF. 2012. “Time for Transformational Change: The Role of the GEF.” www.thegef. 
org/sites/default/files/publications/GEF-vision-Ishii_0.pdf. 

GEF. 2015. GEF 2020: Strategy for the GEF. Washington, DC: Global Environment 
Facility. www.thegef.org/publications/gef-2020-strategy-gef. 

GEF IEO. 2018a. Evaluation of GEF Support for Transformational Change. Washington, DC: 
Global Environment Facility Independent Evaluation Office. 

GEF IEO. 2018b. Impact of GEF Support on National Environmental Laws and Policies in 
Selected Countries 2017. Washington, DC: Global Environment Facility Independent 
Evaluation Office. 

GEF IEO. 2019. Evaluation of GEF Support to Scaling up Impact 2019. Washington, DC: 
Global Environment Facility Independent Evaluation Office. www.gefieo.org/ 
evaluations/scaling. 

IEG. 2014. World Bank Group Support to Electricity Access, FY 2000-2014. Washington, DC: 
Independent Evaluation Group, World Bank Group. https://ieg-worldbankgroup.org/ 
evaluations/world-bank-group-support-electricity-access. 

Rodriguez, H. 2013. “Programa de Energia Eolica en Uruguay (PEEU).” www.the- 
gef.org/sites/default/files/project_documents/2826_UNDP_TE_Informe_Final 
PEEU_Espanol.docx. 


7 Boosting the benefits 
Use of GEF IEO evaluations 


Although evaluations always have a cost, their benefit depends on use: no 
use, no benefit. The Global Environment Facility (GEF) Evaluation Policy! 
spells out the many ways the organization can benefit from making use of its 
evaluations: 


Evaluation can be used to improve the design and performance of a 
planned or ongoing project or program (a formative evaluation); to make 
an overall judgment about the effectiveness of a completed project or pro- 
gram, to ensure accountability; and to generate knowledge about good 
practices.... Evaluation feeds into management and decision-making 
processes regarding the development of policies and strategies; and the 
programming, implementation, and reporting of activities, projects, and 
programs. 


Overall, evaluation contributes to the GEF’s accountability and development 
effectiveness, its institutional learning and evidence-based policy-making, 
and its organizational effectiveness. The GEF uses evaluation results to 
inform its planning, programming, budgeting, implementation, and report- 
ing cycle. Other global development institutions, such as the United Nations 
Development Programme (UNDP) and the World Bank, also use evaluation 
for these purposes. In particular, the GEF comprehensive evaluation (referred 
to as OPS, for overall performance study), completed every four years by the 
GEF Independent Evaluation Office (IEO), illustrates the implementation of 
this dimension of the evaluation policy. 

Evaluation use depends on several factors.? Given the GEF’s context and 
experience, the following are crucial: 


credibility (suitable methods) 

timeliness (availability at the time that decisions are made) 
relevance (responsiveness to demand) 

engagement (of intended users) 

clarity (user-friendly style) 
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We addressed credibility in Chapter 4’s presentation of the IEO’s evaluation 
methods, including frameworks, measures of additionality, and selection of 
evaluation types, all of which are essential for credibility. 


Timeliness 


Of particular importance is the timeliness of reports and their evaluative 
information. As discussed in Chapter 2, which presents the history of eval- 
uation in the GEF, the structure, scope, and development process for the 
comprehensive evaluations evolved and expanded over time as the IEO was 
formed. Most important, the Council members require that the OPS be com- 
pleted, and provide conclusions and recommendations, to meet the timing 
of the replenishment process. During OPS6, the IEO established a process of 
regular feedback to the Council through a variety of report formats as input 
into the seventh four-year replenishment period (GEF-7), July 2018 through 
June 2022. This process included submission of a robust evaluation brief on 
every OPS6 evaluation, for both completed and ongoing projects and pro- 
grams, to the March replenishment meeting.* The comprehensive, timely, 
relevant, and credible inputs into the replenishment process contributed to 
the Council’s approving an increased budget for the IEO. 

The timeliness of completion of the reports and the dissemination of find- 
ings also played a critical role in the GEF’s ability to take up the overall 
strategic recommendations from OPS6 and implement recommendations of 
individual evaluations As indicated in the 2019 Multilateral Organizations 
Performance Assessment Network (MOPAN) independent assessment of 
the GEF, “Timely sharing of evaluation findings with country and regional 
stakeholders has been enhanced by the creation of Evaluation Briefs, origi- 
nally developed to share key messages from OPS6.” 


Relevance and responsiveness 


The OPS comprehensive evaluation is an integral part of the financial 
replenishment process of the GEF. Often, in addition to the IEO’s planned 
work program on institutional, performance, and impact issues, the Council 
requests the IEO to carry out specific studies as inputs into the replenishment 
process. Leading into GEF-7, the Council sought insights into the Integrated 
Approach Pilots that were recently launched in GEF-6 and into GEF govern- 
ance; the IEO responded with evaluations in these two areas. Supplementary 
evaluations like these, which were not part of the original evaluation work 
program, often require the application of a variety of evaluation approaches, 
such as formative reviews in the case of recently approved programs that are 
early in the implementation process. The IEO is also responsible for provid- 
ing the Council with updates on ongoing evaluations. 

For the first GEF-7 replenishment meeting in March 2017, the IEO pre- 
pared a four-page evaluation brief on each ongoing and completed evaluation 
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and provided updated results on interventions for which evaluations had pro- 
gressed substantially. These briefs were very well received. The June 2017 
regular Council meeting included presentation of completed evaluations. 
The IEO prepared OPS6 over the summer of 2017, completing it in time for 
the October 2017 replenishment meeting in Ethiopia that focused on content 
and programming for GEF-7. The IEO was allocated the first hour of the day 
to present the draft report. The discussion on OPS6 lasted five hours at the 
meeting and the replenishment group asked the GEF to reflect the recom- 
mendations from that discussion in the policy and programming directions 
for GEF-7. 

The recommendations included additions of specific references to the IEO 
evaluation on transformational change, early findings from the evaluation of 
the Integrated Approach Pilot programs, and the focal area studies on cli- 
mate change and biodiversity. The programming directions for GEF-7 were 
revised to reference transformational change several times. The IEO evalu- 
ations on safeguards and gender led to the redesign of GEF policies in those 
areas. Other influential evaluations were the Civil Society Organization 
(CSO) Network evaluation, which led to a reform of engagement with the 
network; the private sector evaluation, which led to the development of 
a new private sector strategy in the GEF; and the governance evaluation, 
which led to the formation of an ad-hoc governance working group in the 
GEF Council. Then, at the third replenishment meeting in January 2018 
in Brazil, the replenishment group once again asked the GEF to include 
evaluation recommendations that had been missed in the policy document, 
particularly pertaining to governance. 


Engagement during evaluation 


Throughout the process of evaluation, the IEO engages with numerous 
stakeholder groups. GEF independent evaluations receive feedback from ref- 
erence groups, usually from the GEF Secretariat and Agencies, sometimes 
including the Scientific and Technical Advisory Panel (STAP) and CSO 
Network members, and internal and external peer reviewers. These groups 
review draft reports and make recommendations that are discussed with the 
GEF Secretariat. The IEO presents the evaluations as working documents 
to the GEF Council, and the Council decisions reflect the adoption of the 
recommendations. 

OPS6 involved stakeholder engagement throughout the evaluation. The 
IEO discussed topics and themes for the approach paper with the GEF 
Secretariat and Agencies, operational and political focal points, and civil 
society; we sought inputs throughout the year on process and other issues 
at the expanded constituency workshops with a broad range of stakeholders 
including country focal points, beneficiaries, and civil society representatives. 
The expanded constituency workshops also provided an excellent opportu- 
nity for the IEO to gather information on important issues related to the 
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implementation and sustainability of GEF projects and programs. The indi- 
vidual evaluations that fed into OPS6 also engaged stakeholders throughout 
the evaluation process — from the development of approach papers through 
the sharing of evaluation findings. The findings of the individual evaluations 
and OPS6 were disseminated widely during the year preceding the Assembly, 
contributing to the increased use of the evaluations. Evaluation findings spe- 
cific to the various country constituent units were also presented at the con- 
stituency meetings held as part of the Assembly in Vietnam. 

Implementation of IEO evaluation recommendations is tracked through 
the management action record (MAR) system. The MAR tracks the level 
of adoption of GEF Council decisions that are based on the recommenda- 
tions of the evaluations conducted by the IEO. The MAR serves two pur- 
poses: “(a) to provide Council a record of its decisions based on the evaluation 
reports presented by the GEF IEO, the proposed management actions, and 
the actual status of these actions; and (b) to increase the accountability of 
GEF Management regarding Council decisions” (GEF 2005, p. 3). The level 
of adoption of the decisions is reported every year in the IEO annual perfor- 
mance report. Peer reviews of the IEO have reflected on the inadequacies of 
this process and the need to reform the processes of adopting recommenda- 
tions and the MAR. 


Clarity 


Through a multipronged approach, the IEO ensures the clarity of the eval- 
uative information presented in reports and online, making them easier to 
use for the range of stakeholders. Reports are illustrated with infographics 
and briefs that highlight key messages in four pages or less, as illustrated in 
Figure 7.1. These documents and databases are available on the GEF IEO’s 
user-friendly website, www.gefieo.org. 


IEO evaluation quality and use 


During 2019, IEO conducted a stakeholder survey and knowledge needs 
assessment to obtain feedback on the quality and use of IEO evaluations 
and knowledge products. Ninety-six percent of respondents were satisfied 
with the quality of IEO evaluation reports, and more than 90 percent were 
satisfied with the relevance, usefulness, ease of understanding, transparency 
and clarity of methodology, and objective analysis and findings. Respondents 
also indicated satisfaction with the strong link between conclusions and evi- 
dence and with the timeliness of the evaluations and knowledge products. 
The IEO’s process of stakeholder engagement was satisfactory to 90 per- 
cent of respondents. When we considered responses by position, we found 
that GEF Council members and alternates, GEF Agencies, GEF operational 
and political focal points in the recipient countries, and GEF CSO Network 
members were most satisfied with the evaluations; the GEF Secretariat staff 
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was shared by only a third ofthe con- 
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Changes in policies and practices, 


New policies and practices introduced 
in GEF-6 have benefited the efficiency 
of project programming. The consol- 
idation of the project cycle has been 
appreciated, Tho cancellation policy 
has created incentives for projects to be 
prepared expeditiously for Chiot Exec 
‘tive Officer [CEO] endorsement; it is 
notable that not a single project has 
been canceled since the introduction of 
the policy. and only two projects have 
rocaivod waivers, 

The World Bank and the GEF Secre- 
tariat now have four years’ experionce 
with their harmonization pilot which 
was introduced in November 2012. GEF 
program managers participate in World 
Bonk decision meetings at the con 
cept and approval stages, and in quality 
enhancement reviews. The World Dank 
alco frequently arranges for predect- 
sion meetings to allow more time for 
discussion. In turn, the Bank provides: 
the Council and the Secretariat with its 
‘own documentation at both the Council 


approval and CEO endorsement stages, 
as opposed to using the GEF templates, 
Both 
tomed to this way of aligning the World 
Bank and GEF project cycles and see 
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are, 
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regarding contemporary issues of the 
lobal environment, and operational 
advice to the GEF Secretariat and GEF 
Agencies in preparing and reviewing 
projects and programs. Une STAP panet 
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a $100 bition target for climate finance 
until 2025, Within such a context a cli- 
mate finance scarcity relative to globat 
need, GEF respondents across the 
partnership are concerned both with 
the modesty of donor funding and the 
overall shrinking of donor commitments, 
in an increasingly competitive environ- 
ment. The overall shortage of funding 

is affected by the increased number 

of GEF Agencies, resulting in ower 
funding per Agency. GEF resources are 
modest in comparison to those avai 
able to other environmentat trust nds, 
such as the Green Climate Fund and the 
(Climate Investment Funds. In an elfort 
to heighten their collective effectives 
ness, the pursuit of collaborative rela- 
tionships is supported by 80 percent of 
survey respondents across the OEF. 


Donor commitments. Overall, donors 
have mostly delivered on their financial 
commitments to the BEF. According to 
the GEF Trust Fund Financial Report, 
as of September 30, 2016, 99 percent 

of GEF-6 pledges have been met, with 
Smal arroars stl tingering from prior 
OFF replenishment. As stated by many 
stakeholders across tho GEF, meoting 
donor commitments—and doing so 

on time- is important to maintaining 


widespread confidence in the institu- 

wal mechanism averall, given the 
‘generat environment of funding scarcity. 

Despite the delivery of pledged com- 
mitments, the GEF encountered short- 
falls in available financial resources 
due to foreign exchange volatility. While 
‘such volatility is a normal and dally fea- 
ture of capital markets. the GEF has no 
financial mechanism in place for man- 
aging such risk, This has had detri- 
mental effects on the amount of funding 
available for QEF-6 projects, with impli- 
cations for both countries and Agencies, 
which plan based on donor commit- 
ments. 


Cofinancing policy. Ihe GEF has 
initiated a cofinancing policy intent 

‘on mobilizing financial and other 
resources, The new policy has main- 

tained an aspirational ratio of 6:1 
cotinancing overall for the GEF port 
folio. This ratiowas to be applied at a 
macro porttoto tevet, but was come 
times applied to individual projects. It is 
‘generally recognized that multilateral 
‘development banks (MOBs) are able 
to pursue larger projects with higher 
transactions, allowing them to raise 
higher levels of cofinancing overall. Yet 
the avorage size of GEF projects is too 
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small to be attractive to MDBs, with 
obvious cofinancing implications, Only 
15 percent of survey respondents agree 
that the size of GEF projects is attrac: 
tive for MDBs. 


‘The GEF as a provider of catalytic 
finance, The GEF is considered moder 
ately effective as an enabling and some- 
times catalytic financing organization, 
though mainly at the project level, How- 
ever, there is a widespread and shared 
‘understanding that the GEF is not as 
effective in playing the catalytic rote it 
assumed in early reptonishments, for 
several reasons. 


‘+The environmental finance land- 
scape has changed. ond the GEF is 
no tonger exclusive in its finance of- 
terings. 

‘+ The GEF has spread its funds thin 
among mony more Agencies without 
a corresponding increase in funding. 

‘+The smalt size of current projects on 
average is toss able to facilitate both 
cofinancing and innovation 


Based on survey results, there is 
‘widespread support for the GEF to play 
an important role in experimentation, 
Innovation, and demonstration, There 
are mixed perspectives on the role of 
the GEF in supporting replication and 
scating-up, given the need for large- 
scale resources, 


Private sector financing. Thero is 
general agreement across the part- 
nership that to overcome the paucity of 
financing, the GEF neads to thoroughly 
explore nontraditional donors, including 
the private sector, Broader privato 
sector engagement at the project level 
is also seen as desirable, yet the GEF 
has had limited success in this area, 
Foctors identiiod as constraining the 
GEF's ability to engage with the pri- 
vote sector include the size of project 
tunding, the timited avaltabity of infor- 
mation on GFF offerings and capacities, 
and procosses or mechanisms by which 
to attract private sector financing to the 
diferent focal arcas. Further, the GEF 
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high-quality knowledge-based guidance 
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were the least. The total percentage of respondents who reported a high level 
of satisfaction with the overall quality of reports and timeliness has increased 
by eight percentage points since the last survey, in 2015, while the high level 
of satisfaction with the usefulness of conclusions and recommendations has 
increased by 10 percentage points. 
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In terms of use, 80 percent of respondents reported using the evaluations 
to some extent. More than 75 percent of the respondents use the evaluations 
as reference material, in designing and modifying projects and programs, 
for assessing the performance and results of GEF-supported initiatives, and 
for providing advice to others in the GEF partnership and beyond. Council 
members and alternates use the evaluations to support their assessment of GEF 
performance and to make a case for a specific course of action. Members of 
the GEF partnership, such as the staff of the GEF Secretariat, GEF Agencies, 
GEF operational and political focal points, and GEF CSO Network, reported 
using IEO evaluations to design and modify programs and projects, as ref- 
erence material, to share with others, and to design and modify policies and 
strategies. External stakeholders — including CSOs and national and local 
governmental agencies — use evaluation reports as reference material and for 
sharing with others as an input into the design and modification of their own 
initiatives. 

Evaluation briefs, IEO presentations and workshops, the IEO website, and 
email announcements were rated as the most effective knowledge-sharing 
channels across stakeholder groups. Respondents indicated that, to learn 
about evaluations in the future, they would continue relying on these chan- 
nels and on synthesis notes with lessons across evaluations. Multimedia, 
newsletters, and social media were rated as relatively less effective means of 
sharing knowledge (GEF 2019). 


Challenges 


As pointed out in the MOPAN (2019) independent assessment, “IEO could 
do more to ensure that the lessons derived from evaluations are absorbed by 
key actors.” In particular, MOPAN pointed to the need for more face-to-face 
interactions between the IEO and Council members on the IEO work pro- 
gram and noted the limited capacity of Council members to devote to evalu- 
ations. One possible approach to address this is circulating very brief notes on 
lessons learned from evaluations on different topics to Council members on a 
periodic basis, particularly when new members join the Council. 


Conclusion 


The GEF and its stakeholders make extensive use of IEO evaluation 
reports, briefs, summaries, and other products. To improve interventions 
and understand where, how, and why they succeed or struggle; to support 
decision-making around policies and strategies; and to enhance communi- 
cation with and connection between stakeholders — all of these uses ensure 
that the GEF’s investment in evaluation pays off for progress in achieving 
its mission. Yet there is scope for improvement, to address the challenges 
mentioned above. 
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Notes 


1. See Appendix 1; this is the version approved in June 2020. 

2. There is a vast evaluation literature in which evaluation use is discussed, e.g., 
Rogers (2018), Patton (2008), and Feinstein (2002). 

3. www.gefieo.org/evaluations/ops6-report, see evaluation synopses. 
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8 Lessons from the 
GEF experience 


This final chapter draws on the findings and conclusions of recent evalua- 
tions from the Global Environment Facility (GEF) Independent Evaluation 
Office (IEO) and on the evaluation process to provide relevant lessons for 
environmental and evaluation practitioners. We address two key questions: 
how can we consider the environmental-social-economic interlinkages in 
designing programs and adopting systems-based approaches to evaluate the 
outcomes and impacts of interrelated or integrated interventions while rec- 
ognizing potential synergies and trade-offs? What lessons do the independent 
evaluations! of the GEF offer for the design and evaluation of interventions 
dealing with global environmental issues? 


The context 


We are at a perilous historic juncture: the world is facing the multiple chal- 
lenges of tackling a global pandemic, rising poverty and inequality between 
and within countries, and environmental degradation and global climate 
change that threaten the natural systems we depend on. The United Nations’ 
2030 Agenda for Sustainable Development was launched in 2015 to end pov- 
erty and set the world on a path of peace, prosperity, and opportunity for all 
on a healthy planet, through transformational change. The world adopted the 
agenda’s 17 Sustainable Development Goals (SDGs). However, even before the 
COVID-19 pandemic, progress remained uneven and we were not on track 
to meet the goals by 2030 (United Nations Department of Economic and 
Social Affairs 2020). Although we saw some gains in school attendance, access 
to drinking water, and reduction in the incidence of communicable diseases, 
more people are suffering from food insecurity, the natural environment con- 
tinues to deteriorate at an alarming rate, and dramatic levels of inequality per- 
sist in all regions. COVID-19 has now exacerbated the situation in creating a 
health, economic, and social crisis that is threatening lives and livelihoods and 
erasing the modest progress made in recent years (Naidoo and Fisher 2020). 
The COVID-19 pandemic is clear evidence of the inextricable link between 
the world ecosystem and human health. About two-thirds of all infectious 
diseases in humans have their origins in animals (Coker et al. 2011). Human 
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activities have substantially expanded into previously undisturbed domains 
and have destroyed landscapes, fundamentally altering the interactions 
between animals and humans and affecting the dynamics of pathogen trans- 
mission (Tollefson 2020). Encroaching on new ecosystems lets new viruses 
enter our population. Land mismanagement, habitat loss, overexploitation of 
wildlife, and human-induced climate change have created multiple pathways 
for pathogens to transmit from wildlife to domestic animals and humans, 
impacting our health and well-being. If we do not change our behavior today, 
or if we do not learn from this pandemic, similar multidimensional crises with 
environmental, health, and socioeconomic impacts are likely to recur. 

With the exception of the GEF, no other significant and reliable funding 
resources exist to address biodiversity and environmental issues other than 
climate change. In the course ofits nearly three decades, the GEF has weath- 
ered health and economic crises while supporting a portfolio that simulta- 
neously addresses environmental and human health benefits. Independent 
evaluations of GEF programs offer valuable insights relevant to the current 
pandemic, drawing on lessons learned from implementation. 

Chapters 4—6 presented the findings on the continued relevance of the GEF 
in meeting countries’ needs and assisting them in responding to requirements 
of international environmental conventions. These chapters also explored 
the GEF’s strong track record in delivering overall good project performance 
and driving transformational change. This change includes a focus on gender 
equality and indigenous peoples, with support from institutional systems for 
results and knowledge management. 

GEF interventions in all focal areas — biodiversity, climate change, inter- 
national waters, land degradation, and chemicals and waste — have implica- 
tions for improving human health and other socioeconomic outcomes. The 
shift toward more integrated programs, which cross focal areas to address 
drivers of environmental degradation and achieve transformational change at 
scale, highlights the importance of recognizing the synergies and trade-offs 
between environmental objectives and socioeconomic objectives. The SDGs 
emphasize that the interrelationships among the three pillars of sustainable 
development — social, economic, and environmental — are inseparable. All 
social and economic development rests on the biophysical foundation of the 
natural environment, which we as humans depend upon. This collective evi- 
dence of interconnection clearly points to the need to broaden the environ- 
mental discourse beyond climate change to take full account of biological 
resources, land, forests, water, oceans — and the interlinkages between these 
natural systems and the humans who inhabit them as part of the ecosystem. 
And pandemics must be incorporated into any future sustainable develop- 
ment discussions and plans (Di Marco et al. 2020). Finally, evaluating the 
outcomes and impacts of environmental interventions and understanding 
the interrelationships, synergies, and trade-offs between environmental out- 
comes, socioeconomic outcomes, and the SDGs will require a shift toward a 
more integrated and holistic evaluation framework. 


124 Lessons from the GEF experience 


The past informs the present: relevant 
lessons for the current pandemic 


Following its unique mandate, the GEF has implemented projects commit- 
ted to providing global environmental benefits while also improving human 
health and well-being. The GEF portfolio has not only significant connec- 
tion between the various environmental domains but also profound inter- 
linkages to socioeconomic benefits and human health (GEF IEO 2020c). 


A global wildlife program combats disease transmission 


Of most direct relevance to the current pandemic is the GEF’s program to 
combat illegal wildlife trade. Wildlife trade — both legal and illegal — is a 
transmission pathway that exposes humans to zoonotic diseases (Swift et al. 
2007). The supply chain of wildlife and wildlife products from source areas, 
and transportation through local, national, and international networks to dis- 
tant markets, allows for natural spillover and spread. Unsafe handling and 
other practices allow pathogens to pass from wildlife to humans. And these 
conditions and practices are even more challenging to monitor — and impos- 
sible to regulate — in illegal wildlife trade. Illegal wildlife trade generates 
$26 billion per year, making it the fourth most profitable illegal industry in 
the world. Stopping illegal trade has been a difficult endeavor, and, unfortu- 
nately, as long as demand persists for wildlife products, there will be trade. 

Several GEF-funded projects include activities related to combating illegal 
wildlife trade and address the drivers of biodiversity loss. The GEF’s first 
concerted effort to tackle illegal wildlife trade in a coordinated and compre- 
hensive manner was the Global Partnership on Wildlife Conservation and 
Crime Prevention for Sustainable Development, launched in 2015. Covering 
19 countries in Africa and Asia, the program aims to prevent the extinc- 
tion of threatened species by reducing poaching, curbing trafficking, and 
reducing demand. Its main interventions have included supporting protected 
area management, reducing poaching of target species, engaging commu- 
nities in managing human-wildlife conflict, improving performance across 
the enforcement and criminal justice chain, establishing partnerships, and 
knowledge management. 

The IEO conducted a formative assessment of the program in 2017 (GEF 
IEO 2018c), when the program was in its early stages of implementation. 
The evaluation provided insights into what works, why and under what cir- 
cumstances, for whom, and the extent of the benefits. The lessons learned 
from the formative assessment allowed for timely improvements in program 
implementation. Moreover, the evaluation underscored the need for a com- 
prehensive approach along the entire supply chain — from sources to global 
markets — to contain the spread of illegal trade. 

The COVID-19 outbreak has highlighted the issue of illegal wildlife trade 
and — more broadly — of habitat destruction and biodiversity loss. While 
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maintaining their current global (and local) focus on immediate medical, 
health, and socioeconomic needs, all countries need to be better prepared to 
address and mitigate risks to environmental health and human well-being 
posed by climate change, biodiversity loss, and deforestation and degradation. 


Lessons 


A globally coordinated approach. Often, initiatives are designed 
for single countries, but illegal wildlife trade by its very nature per- 
meates jurisdictional boundaries, requiring cross-boundary strategies 
at the regional and global scales. Illegal wildlife trade, ultimately, is an 
international issue, and combating it requires a more cohesive approach 
with the inclusion of regional initiatives, cross-border activities within 
nationally implemented projects, and coordination among countries on 
transboundary issues. 

Greater attention to a broader spectrum of illegally traded 
species. Most programs are limited to certain threatened and endan- 
gered species and megafaunas. Species coverage needs to be strategically 
expanded to conserve the maximum number of species and save species 
from extinction. The GEF addressed this gap in its follow-on funding 
phase for the Global Wildlife Program by expanding interventions to 
include additional species. Thus, paying attention to more species has 
significance in public health surveillance and in assessing zoonotic risk. 
Appropriate focus on demand. Most interventions typically focus on 
reducing poaching of wildlife in source countries and fighting trafficking 
of illegal wildlife and derived products. Few initiatives focus on reducing 
demand for illegally traded wildlife in Asia, the European Union, and 
the United States, and this effort needs to increase. China has banned all 
forms of illegal wildlife trade and consumption in the context of the cur- 
rent pandemic and in response to growing indications that the COVID-19 
outbreak stemmed from a coronavirus found in wildlife (Vaughan 2020). 
This response highlights the linkage between illegal wildlife trade and 
zoonoses, and the importance of working with countries with the great- 
est demand for wildlife and wildlife products. 

Explicitly addressing political will and corruption. A coordinated 
focus on political will and corruption will ultimately help strengthen 
legislation, law enforcement, and judicial systems in combating illegal 
wildlife trade. 

Enhancing livelihoods. Illegal wildlife trade is driven in large part by 
growing demand from expanding economies in Asia, often facilitated by 
transnational criminal networks. Poverty and absence of sustainable live- 
lihoods drive communities to engage in poaching. Efforts should focus 
on the livelihood security of people living near wildlife, coupled with 
demand reduction strategies including public awareness and behavior 
change campaigns in major market hubs. 
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The health co-benefits of the GEF chemicals and waste portfolio 


Our recent study of the health co-benefits associated with GEF interventions 
offers useful lessons for the current pandemic (Hadjimichael and Batra 2019). 
As a result of globalization and the spread of the chemical manufacturing 
industry, the burden of disease due to pollution is significant and increasing in 
lower- and middle-income countries. Overall, analysis of these projects sug- 
gests that GEF interventions in the chemicals and waste portfolio, primarily in 
artisanal gold mining, have significant health co-benefits. The primary objec- 
tive of the typical GEF chemicals and waste intervention is the elimination of 
harmful chemicals from the environment, which translates into direct health 
impacts — namely, decreased disease burden and mortality. This is not a triv- 
ial finding; the conservative global estimate of observable pollution-related 
deaths in 2015 was 9 million or 16 percent of total deaths. 


Lessons 


* Community health approach to better target interventions to 
the most vulnerable. Hands-on training enabled health-care workers 
to assess cases of mercury poisoning in a timely fashion and to effectively 
manage them, with protocols established to send any unclear diagno- 
ses to the hospital. This success in building accurate community health 
surveillance neatly supplemented efforts to raise awareness of the health 
consequences of mercury use and exposure, mobilizing the community 
and leading to more sustained impact. 

° Enhancing local agency through awareness, education, and 
knowledge dissemination. Personal health impacts are a great moti- 
vating factor for individuals on the ground. Raising awareness, educa- 
tion, and disseminating knowledge help break down exposure pathways 
and inequities in health. Awareness-raising activities around new tech- 
nologies and building community help establish the commitment of 
communities. 

e Industry formalization. Local stakeholder engagement demonstrated 
that the priority for action was formalizing the artisan gold mining 
industry with government support. This step is critical for advancement 
of community-level issues, advocacy, and long-term sustainability. 


Lessons from crisis management during the Ebola crisis 


Our case study of the West Africa Regional Fisheries Program (World Bank 
2017) illustrates the GEF’s continuous support for building resilience in 
Ebola-affected countries to buffer external shocks. The program also demon- 
strates the GEF’s ongoing response to country needs for immediate recov- 
ery and long-term food security through improved governance of fisheries 
resources. 
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The Ebola epidemic, which began in Guinea in late 2013 and spread to 
Sierra Leone and Liberia, affected the means of making a living for millions 
of the poorest and most vulnerable people in the region. The hardest hit 
areas were also among the most agriculturally productive regions of the three 
affected countries. Mandatory quarantine measures and fear of infection kept 
farmers from attending to their fields, resulting in considerable disruption of 
farming activities. Food trade between villages and bordering countries was 
slowed, resulting in food shortages. 

The fishing industry helped feed the population when agricultural lands 
were abandoned during the 2014-2016 Ebola outbreak. Liberia and Sierra 
Leone were part of the initial phase of the GEF-funded regional fisheries 
program (implemented from 2010 to 2016), which aimed to strengthen the 
capacity of recipient countries to govern and manage targeted fisheries, reduce 
illegal fishing, and increase local value added to fish products. 

The project’s investments in fighting illegal fishing have had transforma- 
tive results in Liberia and Sierra Leone. The exclusion of illegal trawlers from 
the six-mile inshore exclusion zone opened space for artisanal fishers, coastal 
communities experienced a change in fish availability, and incomes increased 
(World Bank 2017). During the Ebola outbreak, fish as a food source played a 
more central role as a protein supply for both coastal and inland communities, 
mostly in the form of smoked small pelagic fish transported by road through 
a marketing chain controlled by women fish smokers on the coast. 


Lessons 


e Flexibility and quick adaptation. The GEF recognized that marine 
fish resources represent valuable natural capital that could enhance com- 
munities’ resilience in a time of crisis, if managed properly. When agri- 
culture was impacted during the Ebola crisis, resulting in food shortages, 
the GEF reacted quickly to support the fishing industry through reduc- 
ing illegal fishing and increasing local incomes. 

e Long-term strengthening of governance and building resilience 
through sustainable interventions. Improved governance resulted in 
a reduction in illegal fishing, opening up the space for artisanal fishers. 
Coastal communities experienced greater fish availability and incomes 
increased. To ensure long-term financial sustainability in livelihoods and 
food security of fisher communities, the GEF approved an additional 
grant of $10 million in 2016 to Guinea, Liberia, and Sierra Leone for 
further targeted support of their fisheries sector. All three countries pri- 
oritized conservation of their fish stocks for artisanal fisheries in their 
Ebola recovery plans. The additional financing project (implemented 
from 2017 to 2021) provides incremental funding by supporting a suite 
of interventions that address both immediate recovery strategies and 
long-term responses to strengthening fisheries governance and increas- 
ing resilience. 
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Promotion of innovative solutions for infectious waste 


During the 2014—2016 Ebola outbreak, Guinea, Liberia, and Sierra Leone faced 
challenges in safely disposing of a growing quantity ofinfectious waste. The waste 
generation rate was estimated at 240 L of infectious waste per Ebola patient per 
day (United Nations Development Programme [UNDP 2015]). The infected 
medical equipment and waste had to be properly treated to minimize the risk 
of transmission. However, in many hospitals and community care centers in 
affected countries, Ebola-contaminated waste was burned in barrels, burial pits, 
or low-tech incinerators that emitted dangerous fumes and created toxic ash. 

With support from the GEF and UNDP, environmentally friendly steri- 
lizing equipment was introduced to allow safe disposal of the vast amounts 
of infectious waste generated in treating Ebola patients. This equipment, the 
autoclave, was the first of its kind used in any of the Ebola-affected coun- 
tries. The autoclave uses high temperature and pressure steam to disinfect the 
waste, allowing for safe disposal. Unlike burning or incinerating, it does not 
generate pollutants and has a much smaller carbon footprint. 

The autoclave was designed under a GEF-funded project, Demonstrating and 
Promoting Best Techniques and Practices for Reducing Health-Care Waste to 
Avoid Environmental Releases of Dioxins and Mercury. This program was 
implemented from 2007 to 2012 in Argentina, India, Latvia, Lebanon, the 
Philippines, Senegal, Tanzania, and Vietnam. An important component of the 
project was to develop and disseminate affordable nonburn health-care waste 
treatment technologies that could be built and serviced in Sub-Saharan African 
countries using locally available supplies and skills. After completion of pilot 
activities and testing of the prototype in Tanzania, the new autoclave system 
was produced in partnership with a South African manufacturer, Medi-Clave. 

During the Ebola outbreak, microbiological tests indicated that the virus was 
effectively destroyed by the autoclave system. UNDP immediately launched 
another project that provided 20 autoclaves to the three Ebola-affected coun- 
tries that continued to be used for treating hospital waste in the post-Ebola 
recovery period (UNDP IEO 2013). The autoclave system demonstrated the 
synergy between health-care and environmental sustainability, which can 
lead to overall reduction in harm to both human health and the environment. 

As a result of this project, non-incineration health-care waste treatment 
technologies and mercury-free medical devices were introduced in four 
Sub-Saharan African countries, Ghana, Madagascar, Tanzania, and Zambia, 
to reduce harmful releases from the health sector. 


Lessons 


« Adapting and promoting existing technologies to a new crisis 
situation. The solution of using autoclaves to dispose of hazardous waste 
brought significant and positive change to the Ebola-affected countries 
and has been sustained long after the crisis abated. 
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e Taking risks and allocating resources based on project addition- 
ality to promote new technologies. With severe budget constraints, 
particularly in Ghana and Madagascar, national budget allocations and 
health-care facilities alone could not cover initial capital investments 
and start-up costs to migrate from current unsafe and environmentally 
polluting practices to non-incineration technologies and to phase out 
mercury-containing devices. Funding from the GEF and support from 
project cofinancers was critical for putting environmentally sound prac- 
tices in place for health-care waste management and treatment (UNDP 
2014). (Read more about additionality in Chapter 6.) 


Lessons for achieving transformational 
change and scaling up 


To overcome the unprecedented nature of the pressures faced by the earth’s 
ecosystems, and to achieve impacts at scale, the 2020 GEF strategy (GEF 
2015) recognized the need for promoting transformational change over 
incremental environmental strategies. Transformational interventions are 
defined as engagements that help achieve deep, systemic, and sustainable 
change with large-scale impact in an area of global environmental concern. 
Transformational change, and the process of scaling up, takes place through 
demonstration, mainstreaming, linking, and catalytic effects. The IEO evalu- 
ation highlighted the following criteria for achieving transformational change 
and scaling up impacts (GEF IEO 2017). 


e Ambition. Transformative interventions have ambitious objectives — 
explicit or implicit — to trigger and support a deep, fundamental change 
in addressing a market distortion or systemic bottleneck that is a root 
cause for an environmental issue of global concern. 

e Effective mechanisms. Transformational interventions help to estab- 
lish a mechanism — mainstreaming, demonstration and replication, and/or 
catalytic effects — to scale up and expand the activities supported by the 
intervention. 

e Good-quality implementation and execution. These are important 
drivers of change in terms of the quality of project design, supervision, 
and assistance by the GEF Agency, and the effectiveness of the execut- 
ing agency in performing its roles and responsibilities. Partnerships and 
stakeholder engagement play an important role. 

e Ability to harness market forces. A strong private-sector response is 
important for achieving a fully completed transformation. This suggests 
that where an opportunity exists to harness market forces — by address- 
ing removal of barriers through sound policy, encouraging sustainable 
supply, catalyzing potential demand, and encouraging civil society and 
community participation — it deserves careful attention in the identifica- 
tion and design of an intervention. 
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e Size does not matter. Modest GEF medium-size projects can be just as 
transformational as major, multiphase investment projects. 

¢ Transformation takes time. Transformational interventions and the 
process of scaling up can often take more than five years. Long-term sus- 
tainability of transformational change and scaling up requires establishing 
sustainable sources of financing and strengthening institutional capacities. 


Lessons for long-term sustainability of outcomes 


Although outcome performance is important, the long-term sustainability 
of an intervention’s outcomes is perhaps even more critical. IEO internal 
analysis of the terminal evaluation data shows that projects cutting across 
focal areas have higher outcome ratings but lower sustainability. The Annual 
Performance Report (GEF IEO 2018a) and the country cluster evaluations 
on the small island developing states (GEF IEO 2019), the Africa biomes 
(GEF IEO 2020b), and the least developed countries (GEF IEO 2020a) pro- 
vide useful lessons for enhancing sustainability of outcomes. Inherent in all 
of these lessons is the need to address this objective from the design stage. 


e Demonstrating sustainability takes time. This underscores the 
importance of designing projects with due consideration to measures 
that increase the likely sustainability of outcomes. 

e Plan for financial sustainability. The challenge of financial sustain- 
ability in the least developed countries highlights the importance of 
planning at the design stage to set up viable financial mechanisms and 
measures that can continue to deliver benefits after project completion. 

°. Attend to country considerations. Project design that promotes sus- 
tainability must consider a country’s socioeconomic and political con- 
text, with attention to local conditions and knowledge. Particularly in 
fragile countries, design should include measures and activities designed 
to support — from both financial and institutional standpoints — the con- 
tinuation of outcomes after the project’s completion. 

* Know the influence of synergies and trade-offs. Consideration must 
be given, again, at project design to how synergies and trade-ofts between 
socioeconomic and environmental objectives influence the prospects for 
sustainability in the biomes. This underscores the importance of nexus 
thinking between environmental and socioeconomic objectives and 
between short-term and long-term planning in enhancing sustainability. 


Lessons for partnerships: policies, strong 
institutional systems, and governance 


The GEF partnership delivers its program through 18 implementing agen- 
cies. The effective performance, management, and operation of the partner- 
ship require a set of appropriate policies and procedures. The GEF policies 
and guidance on gender equality, safeguards, and indigenous peoples have 
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advanced the GEF’s efforts in these areas, but gaps exist in the frameworks 
relative to good practice in partner agencies and in implementation. 


e Policies that reflect capacities and deliver guidance. For a partner- 
ship, policies must be based on the capacities of all agencies to implement 
principles and guidelines that reinforce policies. These should be accom- 
panied by practical guidance on project design and indicators, or a list of 
specific requirements to aid in implementation of the policies. 

* Streamlining for private-sector engagement. Partnerships with the 
private sector to address environmental problems have been a challenge for 
the GEF (GEF IEO 2017). Furthering private-sector engagement requires 
eliminating operational restrictions such as long timelines, approval pro- 
cesses, and documentation. The private sector should be engaged as part- 
ner in promoting environmental sustainability more broadly, rather than 
as a source of financing. Institutions such as the GEF should make coun- 
try clients and private-sector stakeholders aware of the opportunities for 
engagement with one another, in finance and beyond. Country recipients 
need to be aware of the private-sector role in green finance, in accessing 
funds beyond the usual GEF grant instruments, or in other opportunities 
for engagement. Private-sector respondents need clarity to help them pre- 
pare for cooperation in partnerships and to help them understand the dis- 
tinct role for multilateral institutions including the GEF in regulatory and 
policy interventions where conditions are still not ripe for investment. 

* Components of good governance. Overall, the GEF partnership is 
effectively governed; concerns remain on matters related to representa- 
tion, efficiency, accountability, and transparency. Lessons from the 
governance study (GEF IEO 2018b) point to key attributes for good 
governance: a high level of trust and goodwill, a sense of common pur- 
pose, a governing instrument that fully and accurately reflects how the 
partnership actually functions, a level of decentralized decision-making, 
transparency at all levels, and an independent chair. 


Lessons for better evaluations 


The GEF evaluation experience, particularly OPS6, offers valuable lessons to 
enhance the quality, credibility, and utility of evaluations. These lessons are 
consistent with the evaluation experience of other global development insti- 
tutions, including the UNDP (2019), World Bank (IEG WBG 2019), and the 
International Fund for Agricultural Development (IFAD 2019). 


Timely delivery and evaluation quality 


Balancing updated information with early delivery 


The GEF provides funding to developing countries largely in the form of 
grants and operates through a four-year funding cycle, at the end of which 
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the donors replenish the fund. Evaluation is critical in the replenishment 
process. On the one hand, conducting the evaluations as late in the process 
as possible will reflect the latest developments in programming and give out- 
comes time to emerge. On the other, decision-makers need evaluation results 
as early as possible to make the best use of them. Balancing these two needs 
requires applying formative and real-time evaluation approaches to provide 
insights into recently designed or ongoing interventions, in addition to the 
traditional ex-post objectives-based evaluation. 


Completion of a quality report requires time and planning 


More than having complete individual evaluations, a good comprehen- 
sive report must allow sufficient time to bring findings together, analyze 
cross-cutting issues, confirm conclusions, and develop recommendations. 
Successful time planning made OPS6 useful to the replenishment process; 
with complete information from 29 evaluations, the analysis and writing 
took five months. Box 8.1 discusses the impact of early and comprehensive 
evaluation planning. 


Box 8.1 


The impact of early and comprehensive evaluation planning 


Planning a comprehensive evaluation early is instrumental to a useful and timely 
evaluation. 

The framework for OPS6 was designed in 2016 with the presentation of the 
approach paper. This document set the stage for the evaluation, laid out a detailed 
timeline, and defined the scope, including the main aspects to be covered in terms 
of results and impact, and organizational, institutional, and governance topics. It 
presented the specific questions that the evaluation would seek to answer and the 
approach and sources for finding those answers. 

Planning allows for early consultation with stakeholders through mechanisms 
such as constituency workshops, which support design of the evaluation to ensure 
its use at key decision-making moments, such as the GEF replenishment. Planning 
also helps with early identification of gaps in sources of evidence. Some may be 
unavoidable due to program implementation timelines, but overall, good planning 
of deliverables and resource use is important. We explore the OPS6 planning pro- 
cess in detail in Chapter 3. 


* Good evaluation practices support evidence-based decisions. 
Adhering to good evaluation practices is very important to quality: devel- 
oping comprehensive concept notes, implementing a rigorous peer review 
and reference group process, pulling a strong evaluation team together, and 
presenting sound conclusions and recommendations grounded in evidence. 
For the GEF seventh replenishment, the endorsement of the recommenda- 
tions in component evaluations at several Council meetings provided an 


evidence-based underpinning for OPS6. 


Lessons from the GEF experience 133 


* Internal efficiency with external support. Evaluators must balance the 
efficiency of working internally with the increased value from external per- 
spectives. Developing the approach paper internally is more efficient than 
developing it with external help, as we discussed in Chapter 3. However, 
regular periodic engagements with an external advisory panel may add sig- 
nificant value to the quality of the report. For example, the OPS6 external 
independent advisory panel played a critical role in ensuring important issues 
were not missed in the report. We shared the main outline and draft findings 
with the panel prior to embarking on writing the first version of the full 
report. This worked out well, but even better would have been their involve- 
ment from the beginning, even with the development of the approach paper. 
The panel’s feedback on the first draft and the final report was extremely 
helpful in sharpening the focus of the conclusions and recommendations. 

* Sharing the knowledge. Facilitating learning through knowledge man- 
agement and dissemination of evaluation findings is as important as pro- 
ducing the report. Translating the evaluation findings into messages for a 
broader audience is worthwhile. Investing in briefs and in a variety of media 
channels, such as presentations, videos, maps, and graphical expositions, was 
helpful in disseminating the main findings of the component evaluations and 
OPS6. Making the results more context specific for various countries was 
also critical for take-up of the findings. Transparency, which requires disclo- 
sure of evaluation reports, is a necessary but not sufficient condition for wide 
knowledge sharing, which requires active dissemination in a broad range 
of ways. 


Political and global factors for evaluators 


Evaluation involves understanding, acknowledging, and managing compet- 
ing stakeholder priorities and the financial environment of the times. 

Like all international initiatives, the GEF must cope with changing trends in 
international cooperation that are reflected in financing priorities. Countries 
and groups of countries promote specific priorities for technical or political 
reasons, and other groups within a partnership like the GEF advocate for 
their own interests. The next GEF replenishment process, in 2021-2022, will 
take place in the midst of recovery from the global stress generated by the 
COVID-19 crisis, and the GEF will be competing for funding with organi- 
zations and programs dealing more directly with health and economic recov- 
ery. Under these circumstances, evaluation informs the political context with 
technical and scientific evidence regarding past performance. The evaluative 
evidence pertains to the achievement of impacts, organizational performance, 
and the effectiveness of institutional and governance arrangements. 


Evaluative evidence for decision-making 


Evaluators must understand internal priorities and help an organization make 
evidence-based decisions. 
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Within the GEF, environmental priorities have had to compete with each 
other for a funding share in what is essentially a zero-sum game. The key 
question remains how to prioritize between the different areas and types of 
intervention within a limited financial envelope. 

Making informed decisions about programming requires knowledge of 
what has proven effective in tackling the specific challenges in an organiza- 
tion’s purview. Systematic evaluation is one of the best ways of generating 
evidence of past performance, factors contributing to results (or lack thereof), 
and the likelihood of achieving desired outcomes and impacts in the most 
effective, efficient, and sustainable manner. 


Ensuring credibility 


To have impact, the evidence presented in an evaluation must be credible — 
and be perceived as such. 

For every GEF evaluation, and particularly for the comprehensive evalua- 
tions, we put extensive effort and consultation into defining the focus, scope, 
approach, and methodologies of the evaluation itself and all of the component 
studies that contribute to it. Our analyses encompass as broad a set of evi- 
dence as possible to allow for triangulation of findings (Carugi 2016). 

We ensure that stakeholders understand the [EO’s independence and 
impartiality in posing the evaluation questions and providing answers to 
them. Independence, methodological rigor, and transparency are important 
dimensions of credibility. 


Emphasizing LEO independence 


As described in Chapter 2, the GEF Instrument specifies the independence of 
the GEF IEO. Our role is to conduct independent evaluations of the GEF’s 
performance and impact and to set and validate the standards for the evalua- 
tion function within the partnership. 

The IEO also provides guidance to the agencies, which are responsible 
for evaluating their own projects and programs. Our guidance addresses the 
design and conduct of the evaluations to ensure use of similar approaches 
to assess key aspects, particularly the achievement of global environmental 
objectives. 


Rigorous methodology and transparency promote credibility 


One way of ensuring rigor is the use of a systematic lens to address eval- 
uation questions, applying the framework of evaluation criteria from the 
Organisation for Economic Co-operation and Development’s Development 
Assistance Committee that address relevance, coherence, effectiveness, effi- 
ciency, impact, and sustainability (Chapter 4 describes how the IEO uses this 
framework). 
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In OPS6, we applied the methods discussed in Chapter 4 as appropriate to 
the various evaluations. We have invested in promoting quantitative rigor in 
our evaluations, while also recognizing the importance of good qualitative 
analysis to interpret and provide contextual explanations for the quantitative 
results. Our mixed-methods approach offsets the limitations of using quanti- 
tative and qualitative methods separately, particularly in explaining outcomes 
and influencing factors within a system of interest. Triangulating results from 
both approaches helps establish integrity and validity of findings and provides 
sound evidence for conclusions and recommendations. 

Throughout the OPS6 evaluation process, we shared the methods, data, 
and approaches for the comprehensive and component evaluations, ensuring 
transparency in all aspects of the evaluation. 


Stakeholder engagement promotes evaluation ownership 


Stakeholder engagement across the evaluation process is vital to benefit from 
stakeholder insights and concerns while also developing their ownership of 
the evaluation. Utilization-focused evaluation and related types of partici- 
patory evaluation (Cousins and Whitmore 1998) emphasize the importance 
of stakeholder involvement and iterative learning through evaluation as the 
foundation of utilization. For OPS6, the approach paper was developed with 
stakeholders in a consultative manner to address their interests. We shared 
the draft report of OPS6 and discussed the findings, conclusions, and rec- 
ommendations prior to finalization and presentation to the Council. We 
also obtained stakeholder feedback throughout the evaluation process on all 
component evaluations and OPS6. The high level of stakeholder engagement 
throughout the OPS6 process contributed to the use and influence of the 
evaluation in the GEF-7 programming directions. 


Putting evaluations to work 


Unless it is used, an evaluation has no value. Utility of evaluations is enhanced 
by structured participation and timeliness. This involves stakeholders at vari- 
ous stages, from the development of the approach paper to the dissemination 
of findings. The OPS6 team discussed topics and themes for the approach 
paper with the GEF Secretariat and Agencies and held constituency work- 
shops with a broad range of stakeholders including government officials, civil 
society representatives, and beneficiaries. 

Further supporting use of IEO evaluations is a built-in feedback loop 
with policymaking, created through the GEF Council’s approval of the IEO 
work program and budget and the IEO director reporting directly to the 
Council. 

The most important element for utility is ensuring on-time delivery, such 
as preparing OPS6 to meet the timing of the replenishment process. As part 
of that process, we disseminated findings of individual evaluations and OPS6 
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during the year leading into the final meetings, increasing the relevance and 
use of the evaluations. 


Applying a systems approach to 
understanding complex relationships 


Our experience from the GEF-7 replenishment process shows that evaluation 
can contribute significantly to devising strategies and designing interventions 
that address both environmental and socioeconomic challenges. Based on a 
rigorous analysis of experiences on the ground, evaluation helps us under- 
stand what works, why, and under what circumstances. Applying a systems 
approach helps address the shortcomings of traditional logic models. These 
will not be adequate for the complex systems in which all interventions take 
place, and linear causality between components of a system is often obscured 
by interdependencies and feedback loops between them. A systems approach 
also helps capture unintended consequences (which are not considered by an 
objectives-based evaluation), particularly in light of the possible trade-offs 
between environmental and socioeconomic goals. 

Evaluating in complex situations requires improved approaches and meth- 
odologies to deal with both environmental and human/societal systems, 
and with their interactions. Evaluation questions must drive the choice of 
appropriate methods, rather than vice versa, as often is the case in impact 
evaluations that emphasize experimental designs. Therefore, use of multi- 
ple methods must become the norm. Evaluations also need to move beyond 
assessing whether individual interventions achieve their targets and outputs, 
and instead focus on the big picture — do the policies, strategies, and inter- 
ventions designed make a dent in the areas that they are intended to benefit? 
In the global environment arena, the challenges are too large to focus on 
activities and outputs; evaluation must focus on outcomes and impacts. The 
SDGs also require us to go beyond monitoring and indicators, and build eval- 
uation into the SDG processes to better understand what is happening and 
why, what causalities foster achievement of goals, and what barriers hamper 
it. Equally important is building in knowledge management so lessons from 
evaluating real-life experiences can inform the development of future poli- 
cies, strategies, and interventions. 


Crises necessitate creativity 


The 2020 global pandemic has made evaluation even more important. While 
the global stress accentuated by the effects of COVID-19 has increased the 
need for evaluations to help in decision-making, the pandemic has restricted 
our ability to do evaluative fieldwork. The COVID-19 crisis has turned eval- 
uators’ lives upside down: suddenly, we can’t visit the sites we are responsible 
for, interview participants in person, or otherwise conduct assessments as 
we normally would. As with much of the world economy, this necessitates a 
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period of unprecedented creativity to ensure that the critical role evaluation 
serves is not neglected for the duration of the crisis. 

One way we are engaging with continuing operations while under pan- 
demic restrictions is by using Quasi-experimental Geospatial Interpolation, 
which combines satellite and other geospatial information with existing sur- 
vey data collected by international organizations, showcased in a recent arti- 
cle in the journal Sustainability (Runfola et al. 2020). In the era of COVID-19, 
with field work halted and working from home as the new normal, such 
methods can help evaluators leverage open-source and readily available data 
to aid in determining the impact of projects. Working with geospatial data 
can help project evaluators maintain flexibility and encourages them to avoid 
designing new and often expensive data tools for every unique evaluation. 
While this method may not be a perfect fit for every evaluation, it provides 
a novel, replicable approach to the estimation of environmental and socio- 
economic outcomes for some types of projects. We also employ tools and 
software such as NVivo for qualitative analysis of large portfolios and seek the 
assistance of networks of national consultants in our client countries. 

For the next comprehensive evaluation, OPS7, we are increasing our plan- 
ning time, engaging our advisory panel at the approach paper stage, and 
have instituted a series of quality control processes within our evaluations 
to enhance quality and validity of our findings. The pandemic has adversely 
affected traditional stakeholder engagement, particularly with respect to our 
beneficiaries, but we are applying a variety of mobile tools and applications 
to garner stakeholder feedback. We are in a time for innovation and greater 
collaboration in evaluations. In partnership with numerous universities and 
consulting firms, we continue to apply a variety of evaluation methods to 
assess the impact of interventions on the human-natural systems nexus. 


Note 


1. “Independent evaluation” is “An evaluation carried out by entities and persons free 
of the control of those responsible for the design and implementation of the devel- 
opment intervention.” www.oecd.org/development/evaluation/2754804.pdf. 


References 


Carugi, C. 2016. “Experiences with Systematic Triangulation at the Global Environment 
Facility.” Evaluation and Program Planning 55: 55—66. 

Coker, R., J. Rushton, S. Mounier-Jack, E. Karimuribo, P. Lutumba, D. Kambarage, 
D.U. Pfeiffer, K. Stark, and M. Rweyemamu. 2011. “Towards a Conceptual 
Framework to Support One-Health Research for Policy on Emerging Zoonoses.” The 
Lancet Infectious Diseases 11: 326-331. 

Cousins, J.B., and E. Whitmore. 1998. “Framing Participatory Evaluation.” In: 
E. Whitmore (ed.), Understanding and Practicing Participatory Evaluation, New Directions for 
Evaluation. San Francisco, CA: Jossey-Bass, 80. 


138 Lessons from the GEF experience 


Di Marco, M., M.L. Baker, P. Daszak, P. De Barro, E.A. Eskew, C.M. Godde, T.D. 
Harwood, M. Herrero, A.J. Hoskins, E. Johnson, W.B. Karesh, C. Machalaba, J. 
Navarro Garcia, D. Paini, R. Pirzl, M. Stafford Smith, C. Zambrana-Torrelio, and 
S. Ferrier. 2020. “Opinion: Sustainable Development Must Account for Pandemic 
Risk.” Proceedings of the National Academy of Sciences of the United States of America 117(8): 
3888-3892. www.pnas.org/content/117/8/3888. 

GEF. 2015. GEF 2020: Strategy for the GEF. Washington, DC: Global Environment 
Facility. www.thegef.org/publications/gef-2020-strategy-gef. 

GEF IEO. 2017. Evaluation of GEF Engagement with the Private Sector 2017. Washington, 
DC: Global Environment Facility Independent Evaluation Office. 

GEF IEO. 2018a. Annual Performance Report 2017. Washington, DC: Global Environment 
Facility Independent Evaluation Office. 

GEF IEO. 2018b. Comparative Advantage, Adequacy of Funding/Financing, Health of the 
Expanded GEF Partnership and Governance Structure 2017. Washington, DC: Global 
Environment Facility Independent Evaluation Office. 

GEF IEO. 2018c. Biodiversity Focal Area Study. Washington, DC: Global Environment 
Facility Independent Evaluation Office. www.gefieo.org/evaluations/biodiversity- 
study-2017. 

GEF IEO. 2019. Small Island Developing States (SIDS) Strategic Country Cluster Evaluation 
(SCCE). Washington, DC: Global Environment Facility Independent Evaluation 
Office. 

GEF IEO. 2020a. Strategic Country Cluster Evaluation of the Least Developed Countries. 
Washington, DC: Global Environment Facility Independent Evaluation Office. 

GEF IEO. 2020b. Strategic Country Cluster Evaluation: Sahel and Sudan-Guinea Savanna 
Biomes. Washington, DC: Global Environment Facility Independent Evaluation Office. 

GEF IEO. 2020c. The GEF Response to the Crisii—What Can We Learn from Evaluation? 
Washington, DC: Global Environment Facility Independent Evaluation Office. www. 
gefieo.org/documents/gef-response-crisis-what-can-we-learn-evaluation. 

Hadjimichael, N., and G. Batra. 2019. A Study on the Health Co-Benefits of GEF Chemicals 
and Waste Focal Area. Washington, DC: Environment Facility Independent Evaluation 
Office. www.gefieo.org/sites/default/files/documents/co-benefits-cw-study-2019.pdf. 

IEG WBG. 2019. IEG Annual Report 2019: Transforming Evidence into Better Outcomes. 
Washington, DC: Independent Evaluation Group, World Bank Group. https://ieg. 
worldbankgroup.org/evaluations/ieg-annual-report-2019-transforming-evidence- 
better-outcomes. 

IFAD. 2019. 2019 Annual Report on Results and Impact of IFAD Operations. Rome: 
IFAD Independent Office of Evaluation. www.ifad.org/en/web/ioe/evaluation/ 
asset/41391061. 

Naidoo, R., and B. Fisher. 2020. “Sustainable Development Goals: Pandemic Reset.” 
Nature 583: 198—201. 

Runfola, D., G. Batra, A. Anand, A. Way, and S. Goodman. 2020. “Exploring the 
Socioeconomic Co-Benefits of Global Environment Facility Projects in Uganda Using 
a Quasi-Experimental Geospatial Interpolation (QGI) Approach.” Sustainability 12(8): 
3225. www.mdpi.com/2071-1050/12/8/3225. 

Swift, L., P.R. Hunter, A.C. Lees, and DJ. Bell. 2007. “Wildlife Trade and the 
Emergence of Infectious Diseases.” Ecohealth 4(1): 25. www.ncbi.nlm.nih.gov/pmc/ 
articles/ PMC7087654/. 

Tollefson, J. 2020. “Why Deforestation and Extinctions Make Pandemics More Likely.” 
Nature 584: 175-176. 


Lessons from the GEF experience 139 


UNDP. 2014. Reducing UPOPs and Mercury Releases from the Health Sector in Africa. 
New York, NY: UnitedNations. www.thegef.org/sites/default/files/project_documents/ 
5-19-2014_1D4611_ProjeDoc_0.pdf. 

UNDP. 2015. Assessment and Recommendations Regarding Management of Ebola- Contaminated 
Waste. New York, NY: United Nations. https://noharm-global.org/sites/default/files/ 
documents-files/3127/Report%20to%20WHO%20WASH%20and%20Geneva%20 
on%20Ebola%20final.pdf. 

UNDP. 2019. United Nations Development Programme Annual Report 2019. New York, 
NY: United Nations. www.undp.org/content/undp/en/home/librarypage/corporate/ 
annual-report-2019.html. 

UNDP IEO. 2013. Global Project on Demonstrating and Promoting Best Techniques and 
Practices for Reducing Health-Care Waste to Avoid Environmental Releases of Dioxins 
and Mercury. Terminal Evaluation Report. www.thegef.org/sites/default/files/ 
project_documents/1802_UNDP_TE_00058547_-_POPs_MedicalWaste_-_ 
EoPEvaluation_-_Summary_LessonsLearnt.docx. 

United Nations Department of Economic and Social Affairs. 2015. “Transforming Our 
World: The 2030 Agenda for Sustainable Development.” https://sustainabledevelop- 
ment.un.org/post2015/transformingourworld. 

United Nations Department of Economic and Social Affairs. 2020. The Sustainable 
Development Goals Report 2020. New York, NY: United Nations. https://unstats.un.org/ 
sdgs/report/2020/. 

Vaughan, A. 2020. “Coronavirus: China Wildlife Trade Ban Could Become Law Within 
Months.” New Scientist. www.newscientist.com/article/2239559-coronavirus-china- 
wildlife-trade-ban-could-become-law-within-months/. 

World Bank. 2017. Terminal Evaluation of First Phase in Support of the West Africa Regional 
Fisheries Program (WARFP). Washington, DC: World Bank. www.thegef.org/sites/ 
default/files/project_documents/3558%2520ICR_1.pdf. 


Appendix 1 
Global Environment Facility 
Evaluation Policy 


gef GLOBAL ENVIRONMENT FACILITY 
INVESTING IN OUR PLANET 


GEF/ME/C.56/02/Rev.01 
June 13, 2019 

56th GEF Council Meeting 
June 11th -13, 2019 
Washington, D.C. 

Agenda Item 11 


The GEF Evaluation Policy 
(Prepared by the Independent Evaluation Office of the GEF) 


Recommended council decision 


The Council, having reviewed document GEF/ME/C.56/02, “The GEF 
Evaluation Policy”: 


1 Approves the proposed GEF Evaluation Policy contained in Section 
2 of this document, which supersedes the 2010 Policy on Monitoring 
and Evaluation as it relates to evaluation (Council document GEF/ 
ME/C.39/6/Rev.1).” 

2 Authorizes the GEF Independent Evaluation Office to proceed with its 
dissemination and implementation. 
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Section 1: Background 


1 In May 2014, the GEF Independent Evaluation Office (IEO) recom- 
mended an update to the GEF Monitoring and Evaluation Policy to the 
GEF Council, to ensure its continued relevance in the context of the 
evolution of the GEF partnership since 2010.' In October 2015, the IEO 
analyzed all GEF Council decisions taken since the last M&E Policy was 
issued in 2010. This analysis identified several gaps in the 2010 Policy 
including gender and safeguard standards, and program level evaluation, 
among others.” 

2 Based on international good practice standards among organizations 
including global partnerships and multilateral development banks, the 
2010 M&E Policy has been split into two separate policies, one address- 
ing monitoring, and the other evaluation. An Evaluation Policy defines 
the core principles, criteria, and the governance structure (key roles and 
responsibilities) of an organization’s evaluation function. The monitoring 
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and reporting function is an integral part of an organization’s operations, 
which is consistent with its project cycle, and defines clear roles, respon- 
sibilities, and milestones on the provision of information and data for 
reporting. 

3 The next section of this document presents the revised GEF Evaluation 
Policy that would supersede the provisions for evaluation contained in 
the 2010 M&E Policy. This policy has been prepared with engagement 
of stakeholders across the GEF partnership. A separate GEF Monitoring 
Policy has been prepared by the GEF Secretariat. The IEO and the 
Secretariat have consulted to maintain continuity and coherence between 
the provisions of the two policies. 

4 The main updates to the GEF Evaluation Policy include: 


a Introducing the principle that evaluation in the GEF should apply a 
gender-responsive approach 

b Introducing the requirement that terminal evaluations of GEF projects 
and programs should report on the GEF’s additionality using the eval- 
uative approach provided by the GEF IEO? 

c Introducing the requirement that program evaluation should assess 
the coherence between program and child project theories of change, 
indicators, and expected/achieved results 

d Establishing the principle that program evaluation should measure and 
demonstrate program value added over the same level of investment 
made through comparable alternatives 

e Clarifying evaluation responsibilities for jointly implemented projects 

f Introducing a requirement to collect (1) socio-economic co-benefits 
data, (2) sex-disaggregated and gender sensitive data, and (3) geographic 
coordinates of project sites whenever available/possible 


Section 2: The GEF Evaluation Policy 
Key terms 


Agency fee: the financing provided to a GEF partner Agency in connection 
with a GEF project or program 

CEO Approval: the approval of a fully developed medium-sized project or 
enabling activity by the GEF CEO 

CEO Endorsement: the endorsement of a fully developed full-sized project 
by the GEF CEO 

Child project: a project that forms part of a program, as set out in a program 
framework document 

Co-financing: financing additional to GEF project financing, and that sup- 
ports implementation of a GEF-financed project or program and the achieve- 
ment of its objectives 
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Evaluation: Evaluation is the systematic and impartial assessment of planned, 
ongoing, or completed activities, projects, programs in specific focal areas or 
sectors, policies, strategies and their implementation, or other topics relevant 
to the GEF partnership and organization* 

Full-sized project: a project with GEF project financing exceeding US$2 
million 

GEF additionality: the additional effects (both environmental and other- 
wise) that can be directly associated with a GEF-supported project or program 
GEF Agency: an agency eligible to request and receive GEF resources 
directly for the design, implementation, and supervision of GEF projects and 
programs 

GEF Instrument: Instrument for the Establishment of the Restructured 
Global Environment Facility, effective July 7, 1994, as amended 
Gender-sensitive indicator: an indicator that can be used at various levels 
to monitor and report on socioeconomic and gender-sensitive changes over 
time 

GEF-financed activity (or intervention): any programmatic approach, 
full-sized project, medium-sized project, or enabling activity financed from any 
GEF-managed trust fund, as well as regional and national outreach activities 
GEF Operational Focal Point: nominated by the recipient country, the 
GEF Operational Focal Point ensures that GEF proposals and activities in 
the country are consistent with country priorities and the country commit- 
ments under global environmental conventions; identifies project ideas to 
meet country priorities; endorses project proposals; facilitates broad based 
in-country consultations on GEF operational matters; and provides feedback 
on GFF activities, including implementation of projects? 

Global Environmental Benefits: these relate to international conventions 
and commitments the GEF is mandated to serve. GEF projects must demon- 
strate that the project activities are delivering global environmental benefits 
Goal: a higher-order objective to which a GEF-financed project or program 
is intended to contribute 

Knowledge Management: the process by which organizations within the 
GEF partnership generate value and improve performance from their intel- 
lectual and knowledge-based assets 

Impact: the positive and negative, primary and secondary long-term 
effects produced by a project or program, directly or indirectly, intended or 
unintended 

Indicator: a quantitative or qualitative factor or variable that provides a 
simple and reliable means to measure achievement, to reflect the changes 
connected to a project or program, or to help assess the performance of an 
organization 

Lead Agency: The Agency that coordinates all activities under a program 
Medium-sized project: a project with GEF project financing of up to 
US$2 million 
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Midterm review: an assessment of a project or program’s performance and 
results carried out for adaptive management purposes at the midpoint of a 
project or program’s intended duration 

Monitoring: a continuous or periodic function, carried out by project or 
program management, that uses a standardized and systematic process of col- 
lecting and analyzing data on specific indicators to provide decision-makers 
and management of a GEF-financed activity with information on progress in 
the achievement of objectives and in the use of allocated funds 

Outcome: an intended or achieved short- or medium-term effect of a pro- 
ject or program’s outputs 

Output: a product or service that results from the completion of activities 
implemented within a project or program 

Portfolio: a subset of projects focusing on a specific theme, GEF focal area, 
geographic region, country, or GEF Agency 

Program: a coherent set of interventions designed to attain specific global, 
regional, country, or sector objectives, consisting of a variable number of 
child projects 

Program’s added value: the additional results brought in by the GEF 
funding delivered as a program compared with either a pre-existing or a 
hypothetical set of stand-alone full- and/or medium-sized projects or other 
comparable alternatives 

Program framework document: the document that sets forth the concept 
of a program that is proposed for GEF financing 

Result: Include intervention outputs, outcomes, progress toward longer-term 
impact including global environmental benefits, and should be discernible/ 
measurable 

Stakeholder: an individual or group that has an interest in the outcome of 
a GEF project or program or is likely to be affected by it, such as local com- 
munities, indigenous peoples, civil society organizations, and private sec- 
tor entities; stakeholders may include national project or program executing 
agencies, or groups contracted to conduct activities at various stages of the 
project or program 

Stakeholder engagement: a process that begins with stakeholder identifi- 
cation and analysis, and includes planning; disclosure of information; consul- 
tation and participation; monitoring, evaluation, and learning throughout the 
project cycle; addressing grievances; and ongoing reporting to stakeholders 
Terminal evaluation: evaluation of a project or program’s design, perfor- 
mance, and results carried out at the end of implementation 


Abbreviations 

CEO Chief Executive Officer 
CSO civil society organization 
GEF Global Environment Facility 


IEO Independent Evaluation Office 
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NGO non-governmental organization 

OFP Operational Focal Point 

SMART specific, measurable, attributable, relevant, and time-bound 
STAP Scientific and Technical Advisory Panel 


I. Evaluation in the GEF 


1 


This Policy sets out the guiding principles and minimum 
requirements for evaluation across the Global Environment 
Facility (GEF) partnership and for all GEF-financed activities. 
This Policy is aligned with the GEF Instrument, which provides that 
“the Council shall...ensure that GEF policies, programs, operational 
strategies and projects are monitored and evaluated on a regular basis.”® 
This Policy explains the concept, role, and use of evaluation 
within the GEF. It defines the institutional framework and respon- 
sibilities of stakeholders; and establishes the requirements for how pro- 
jects and programs funded by the GEF should be evaluated in line with 
international principles, norms, and standards. The policy is designed 
to provide clarity on evaluation in the GEF. Revisions will be made 
when major changes in the GEF occur, affecting the evaluation function. 
In addition, to ensure that the Policy remains relevant to evolving cir- 
cumstances and continues to conform to international principles, norms, 
and standards, it will be periodically reviewed and updated as necessary. 
Any proposals for changes in the Policy will be presented by the GEF 
Independent Evaluation Office (IEO) to the Council for decision after 
consultation with stakeholders. 

Evaluation helps the GEF to become more effective in its pursuit of global 
environmental benefits. Evaluation in the GEF has the following two 
overarching objectives: 


a Promote accountability for the achievement of GEF objectives 
through the assessment of results, effectiveness, processes, and perfor- 
mance of the partners involved in GEF-financed activities; GEF results 
are evaluated for their contribution to global environmental benefits. 

b Promote learning, feedback, and knowledge sharing on results 
and lessons learned among the GEF and its partners as a basis for 
decision-making on projects, programs, program management, pol- 
icies, and strategies; and to improve performance. 


The broader framework of GEF Policies are consistent with this 
Policy.’ The evaluation sections of GEF Policies are in full alignment 
with the standards and minimum requirements of this Policy. 

Monitoring and evaluation are two distinct, yet interconnected 
functions in the GEF. Monitoring, conducted as a systematic manage- 
ment function, informs whether a project, program, policy implementa- 
tion, or the organization is achieving its intended objectives as planned. 
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Evaluation is a systematic and objective assessment of an on-going or 
completed project, program, or policy, its design, implementation, effec- 
tiveness and results. Evaluation provides information that is credible 
and useful, enabling the incorporation of lessons learned into the deci- 
sion-making process of both recipients and donors. Evaluation verifies and 
uses monitoring data in its analyses as one of its sources of information. 
Evaluation also provides evidence on how changes are taking place, and 
the strengths and weaknesses of the design of the projects, programs, or 
operational strategies embedded in the GEF results frameworks. 

6 This Policy discusses monitoring only in its interrelations with 
evaluation. A separate GEF Monitoring Policy contains the basic provi- 
sions, standards, and requirements for monitoring in the GEF partnership. 


1. Background 


7 The GEF isa financial mechanism for international cooperation based on 
a partnership. It provides new and additional grant and concessional fund- 
ing to meet the incremental costs of measures to achieve agreed global 
environmental benefits in five focal areas. These are biological diver- 
sity, climate change mitigation and adaptation,® international waters, 
land degradation, and chemicals with sustainable forest management 
cross-cutting relevant focal areas—in developing countries and countries 
with economies in transition. In addition, the Integrated Approach Pilots 
in GEF-6, and the Impact Programs introduced in GEF-7, also use focal 
area allocations in an integrated manner for systems change. The GEF 
Council provides strategic and policy direction in these five focal areas, 
taking into account guidance from the conferences of the parties to the 
relevant global environmental conventions. 

8 The GEF Instrument requires the GEF Council to ensure that 
GEF projects, programs, policies, and operational strategies are 
monitored and evaluated on a regular basis. Meeting this require- 
ment entails feedback to the GEF decision-making processes at the insti- 
tutional, policy, program, and project levels. Corporate monitoring and 
evaluation in the GEF is based on regular reporting for internal man- 
agement purposes to the GEF Council, in support of decision making, 
policy making, and accountability (Figure A1.1). 

9 Evaluation allows the GEF to track progress in fulfilling its mis- 
sion of delivering global environmental benefits. GEF projects and 
programs are more likely to capitalize on their innovative and catalytic 
role when they are fully integrated with the GEF results frameworks and 
where management activities as well as post-completion decision mak- 
ing are informed by comprehensive and relevant evaluative evidence. 
Evaluation strengthens the GEF partnership and encourages ownership 
of GEF projects and programs, all of which are essential principles of 
GEF operations and policies. The GEF emphasizes the quality of its 
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Semi-Annual Evaluation Report 
Comprehensive Evaluation of the GEF 
Annual work program and budget 


Programming documents, results frameworks and core indicators 
Annual Portfolio Monitoring Report 
Management responses to evaluations 


GEF 
Independent Evaluation Office 


Agency h So GEF Secretariat 
evaluation units 


Corporate evaluations 


Independent project and Project and program implementation reports 
program evaluations Projectiand progran Core indicators (GEF-6 onwards) 
terminal evaluations Tracking Tools (GEF-5 and earlier) 
Midterm reviews 
Agency GEF 


coordination units 


| Project and program data and information 


GEF Projects and 
Programs 


Figure A1.1 Simplified flowchart of monitoring and evaluation reporting in the GEF 


evaluation function and ensures that evaluation findings are dissemi- 
nated widely. The Independent Evaluation Office of the GEF (IEO) is 
directly accountable to the GEF Council and has the mandate to report 
on the performance and effectiveness of GEF projects and programs. 
In addition, the evaluation units of the Agencies are responsible for the 
conduct of project and program terminal evaluations in compliance with 
this policy and in accordance with the terminal evaluation guidelines. 

10 The Policy shall be operationalized through guidelines on spe- 
cific issues and standards developed by the IEO in consulta- 
tion with partners. The IEO is authorized to publish and revise such 
guidelines, as required, in line with this Policy. The Policy and related 
guidelines will be shared with the GEF partners and the public through 
the IEO website. 


2. Evaluation: Purposes, use, and types 


11 Definition. Evaluation is the systematic and impartial assessment of 
planned, ongoing, or completed activities, projects, programs in spe- 
cific focal areas or sectors, policies, strategies and their implementa- 
tion, or other topics relevant to the GEF partnership and organization. 
In case of activities, projects, and programs, it aims at determining the 
relevance, efficiency, effectiveness and sustainability of the interventions 
and contributions of the involved partners. An evaluation provides evi- 
dence-based information that is credible, reliable, and useful, enabling 
the timely incorporation of findings, recommendations, and lessons into 
decision-making processes. In the context ofthe GEF, the evaluation func- 
tion aims at assessing the relevance, efficiency, effectiveness, and—where 
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12 


13 


14 


feasible—sustainability of GEF interventions in the context of their con- 
tribution to global environmental benefits in GEF focal areas at the local 
and global levels. 

Purpose. The purposes of evaluation include understanding why, how, 
and the extent to which intended and unintended results are accrued, 
and their impact on stakeholders. Evaluation is an important source of 
evidence of the achievement of results and institutional performance 
and contributes to knowledge and organizational learning. It serves as 
a driver of change and plays a critical role in supporting accountabil- 
ity. Evaluation can be used to improve the design and performance of 
a planned or ongoing project or program (a formative evaluation); to 
make an overall judgment about the effectiveness of a completed project 
or program, to ensure accountability; and to generate knowledge about 
good practices. Evaluation differs from other oversight mechanisms such 
as investigation and audit, which focus on the adequacy of management 
controls; compliance with regulations, rules, and established policies; 
and the adequacy of organizational structures and processes. 

Use. Evaluation feeds into management and decision-making processes 
regarding the development of policies and strategies; and the program- 
ming, implementation, and reporting of activities, projects, and programs. 
Thus, evaluation contributes to institutional learning and evidence-based 
policy making, accountability, development effectiveness, and organiza- 
tional effectiveness. It informs the planning, programming, budgeting, 
implementation, and reporting cycle. It aims to improve the institutional 
relevance and achievement of results, optimize the use of resources, and 
maximize the impact of the contribution provided. 

Types. Within the context of the GEF, project/program level termi- 
nal evaluations relevant to the Agencies interventions are conducted by 
the Agencies. Agencies also conduct mid-term reviews of projects, and 
sometimes carry out impact evaluations. The IEO validates the terminal 
evaluations of GEF projects from all Agencies, and conducts a spectrum 
of evaluations covering thematic issues, focal areas, institutional poli- 
cies and programs, and the comprehensive evaluation of the GEF, all of 
which cut across the entire GEF partnership. The main types of evalua- 
tions in the GEF include, but are not limited to the following: 


a Project evaluations at the completion of the project implementation 
(terminal evaluation), mainly conducted by GEF Agencies. 

b Program evaluations of a coherent set of interventions to attain spe- 
cific global, regional, country, or sector objectives; these include eval- 
uations of the GEF focal areas, programmatic approaches, and GEF 
corporate programs. 

c Performance evaluations of the GEF’s portfolio of completed pro- 
jects to assess the relevance, efficiency and effectiveness in delivering the 
expected results. 
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d Country and country cluster evaluations of one or more Agencies’ 
portfolio of projects and activities, and the assistance strategy behind 
them, in a country or cluster of countries. Strategic cluster evaluations 
focus on a limited set of strategic issues across country portfolios. 

e Process evaluations of the functioning of participating organiza- 
tions, instruments, mechanisms, and management practices; these 
include evaluations of institutional and procedural issues across GEF 
focal areas and assessments of experience with GEF policies, criteria, and 
procedures. Process evaluations could be conducted during the design 
or implementation of an intervention. 

f Impact evaluations of the long-term effects produced by an inter- 
vention, intended or unintended, direct or indirect. Impact may be 
assessed at the project, program, portfolio, ecosystem, or country level, 
and includes global environmental benefits. 

g Thematic evaluations ofa selection of interventions addressing a spe- 
cific theme, issue or focal area across the GEF portfolio; these include 
evaluations that assess cross-cutting issues. 

h Comprehensive Evaluation of the GEF, previously the Overall 
Performance Study (OPS), informing the GEF replenishment cycles, 
and addressing performance and results in terms of global envi- 
ronmental benefits of GEF projects and programs, as well as GEF 
institutional arrangements, policies, strategies, and priorities. The 
evaluations referred to in a—h, above, feed into the comprehensive 
evaluations. 


3. Follow-up to IEO evaluations 


15 


16 


All evaluation reports presented to the GEF Council by the IEO 
require a management response. The GEF Chief Executive Officer 
(CEO) coordinates the preparation of the management response with 
Agency stakeholders for GEF Council consideration, in response to the 
recommendations of each evaluation report. Management responses 
should clearly indicate whether management accepts, partially accepts, 
or rejects the IEO evaluation recommendations, and explain the reasons. 
The IEO may comment on the management response to ensure recom- 
mendations have been addressed. The GEF Agencies ensure that recom- 
mendations from IEO evaluations that are relevant and/or apply to them 
are considered for decision making and action within the Agencies. The 
Council discusses and reviews the evaluation reports, the recommended 
actions, and the management responses; takes any necessary decisions on 
the recommendations; and gives guidance to the GEF and Agencies on 
policies or on action plans with specific time frames. 

There is systematic follow-up to the implementation of evalua- 
tion recommendations through the Management Action Record. 
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There is also systematic follow-up on implementation of the evaluation 
recommendations accepted by management and/or the GEF Council, 
with periodic review and follow-up on their status. In consultation with 
the appropriate GEF partners, the IEO reports to the Council on the 
follow-up of Council decisions related to evaluation recommendations; 
these decisions and follow-on actions are compiled in its Management 
Action Record (MAR) and are provided to the Council on an annual 
basis in the Annual Performance Report. 


4. Knowledge sharing from evaluations 


17 


18 


19 


20 


Evaluation contributes to knowledge building and organiza- 
tional improvement. Findings and lessons must be made accessible to 
target audiences in a user-friendly way. Evaluation reports must have a 
dissemination strategy tailored to the audience of each specific report; 
the strategy must be described in the relevant evaluation approach paper 
and in the terms of reference. 

For the purposes of this Policy, knowledge management is the process 
by which organizations within the GEF partnership generate value and 
improve performance from their intellectual and knowledge-based assets. 
Knowledge sharing enables partners to capitalize on lessons learned by 
gaining insight and understanding from experience, and by applying 
this knowledge to generate new knowledge. Knowledge management 
helps the GEF create and transform knowledge into action, innovation, 
and change. Knowledge management is closely linked to performance 
enhancement. 

The main purposes of knowledge creation and sharing of information 
from GEF evaluations are to: 


a Promote learning through better outreach to the project, program, 
and country levels by providing easily accessible learning products; 

b Promote the application of lessons learned to improve the perfor- 
mance and impact of GEF activities; and 

c Promote feedback to improve the development of projects and 
programs. 


Knowledge creation and sharing from evaluation supports policy 
making by building a comprehensive body of evidence, lessons 
learned, and good practices. Evaluation is closely linked to policy 
making, more informed management, and decision making for strategic 
planning. Evaluations could provide an effective way to improve the per- 
formance and impact of policies, programs, and projects, especially when 
they are conducted at the appropriate time and focus on issues of concern 
to policy makers and managers. 


21 
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Lessons from evaluations should be made available to stake- 
holders directly involved in project and program formulation 
and implementation for improved effectiveness. GEF partners are 
expected to seek out adaptive and interactive ways of disseminating find- 
ings from evaluations to a wide audience, including within and across 
GEF Agencies and the GEF partnership, environmental entities, aca- 
demia, research institutions, civil society, and the general public. When 
lessons and findings are shared widely, evaluations have the potential to 
increase awareness of the importance of global environmental benefits, 
confidence in GEF work, and leverage support. 


II. Evaluation: Norms, principles, criteria, 


22 


23 


and minimum requirements 


Evaluation in the GEF context is guided by internationally rec- 
ognized principles, norms, and standards. Specifically, the GEF 
and its Agencies refer to those principles, norms, and standards produced 
by the United Nations Evaluation Group (UNEG), the Development 
Assistance Committee of the Organization for Economic Co-operation 
and Development (OECD-DAC), and the Evaluation Cooperation 
Group (ECG) of the multilateral development banks (MDBs). Although 
there is general agreement around internationally recognized norms 
and standards, there is also a divergence resulting from the diverse goals 
and objectives of the individual Agencies. These differing goals lead to 
differences in emphasis and differences in the application of standards 
across Agencies. Guided by international norms and standards, the GEF 
Evaluation Policy considers these differences and establishes a set of key 
principles and criteria common across the GEF partnership. The Policy 
also establishes four mandatory minimum requirements Agencies must 
follow in conducting evaluations for GEF-financed activities. 
Evaluation must be an explicit part of planning and budget- 
ing of GEF-financed activities. A key international norm concerns 
the adequate provision of resources to enable evaluation functions to 
operate effectively. In line with this norm, the costing and budgeting 
of evaluation activities are addressed, as appropriate, in the budgetary 
planning of the IEO, the Agency fee system, and project and program 
budgets. This includes any additional financial implications of address- 
ing the minimum requirements and responsibilities of this Policy. All 
GEF Agencies receive project/program allocations and Agency fees, to 
be used according to the provisions contained in the Fee Policy for GEF 
Agencies. Project allocations cover the requirement for monitoring and 
evaluation, including terminal evaluations. Consistent with good prac- 
tice, budgetary resources should be allocated separately for monitoring 
and evaluation activities. 
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24 Evaluation in the GEF context is guided by internationally rec- 
ognized principles. The principles below are internationally recog- 
nized professional standards that should be applied in all evaluations of 
GEF-financed activities: 


a Independence. Evaluations must be conducted independently from 
both the policy-making process and from the delivery and management 
of assistance. Evaluation team members should not have been person- 
ally engaged in the activities to be evaluated or have been responsible 
in the past for the design, implementation, or supervision/mid-term 
review of the project, program, or policy to be evaluated. Where eval- 
uations are financed and/or managed by the GEF coordination units in 
GEF Agencies, these Agencies should ensure the behavioral independ- 
ence of the evaluators. Behavioral independence requires that even 
if the evaluator is contracted by the Agency operational unit whose 
project or program is being evaluated, the unit should not interfere 
with or influence the process, or the interpretation and reporting of 
the evaluation findings. 

b Credibility. Evaluations must be credible and based on reliable data 
and observations. Evaluation reports should reflect consistency and 
dependability in data, findings, judgments, and lessons learned, with 
reference to the quality of the instruments, procedures, and analysis 
used to collect and interpret information. 

c Utility. Evaluations must serve the information needs of intended 
users. Partners, evaluators, and units commissioning evaluations 
should endeavor to ensure that the work is well informed, relevant, 
and timely, and that it is clearly and concisely presented so as to be of 
maximum benefit to intended users. Evaluation reports should pres- 
ent the evidence, findings, issues, conclusions, and recommendations 
in a complete and balanced way. They should be both results- and 
action-oriented. 

d Impartiality. Evaluations must give a comprehensive and balanced 
presentation of the strengths and weaknesses of the project, program, 
policy, strategy, or organizational unit being evaluated. The evalua- 
tion process should reflect impartiality at all stages and consider the 
views of all stakeholders. Units commissioning evaluations should 
endeavor to ensure that the selected evaluators are impartial and 
unbiased. 

e Transparency. An essential feature at all stages of the evaluation pro- 
cess, transparency involves clear communication concerning decisions 
for the program of work and areas for evaluation, the purpose of the 
evaluation, the criteria applied, the evaluation approach and methods, 
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and the intended use of the findings. Documentation related to evalu- 
ations must be freely available, easily accessible, and readable for trans- 
parency and legitimacy. 

f Integrity. Evaluations must provide due regard to the welfare, beliefs, 
and customs of those involved or affected, avoiding or disclosing any 
conflict of interest. Evaluators must respect the right of institutions 
and individuals to provide information on the facts confidentially. 
Evaluators must honor the dignity, well-being, and self-worth of indi- 
viduals and acknowledge the influence of culture within and across 
groups. If evidence of wrongdoing is uncovered, the evaluator or man- 
ager shall report such cases discreetly to the IEO Director, who will 
take appropriate action such as informing the investigative body of the 
relevant Agency. Integrity requires that management and/or commis- 
sioners of evaluations communicate clearly, remain open to the find- 
ings, and do not allow vested interests to interfere with the evaluation. 

g Participation. GEF evaluations must be inclusive, so that the diverse 
perspectives and the values on which they are based as well as the 
types of power and consequences associated with each perspective are 
represented. Evaluation teams should interact with representatives of 
all the stakeholders involved in the project, program, or topic being 
evaluated. The participation of in-country stakeholders, including 
the GEF Operational Focal Point (OFP) as well as other stakehold- 
ers such as project managers and civil society organizations (CSOs) 
involved in project implementation, and project beneficiaries should 
be ensured. 

h Gender equality. Gender equality and women’s empowerment is 
a strategic and operational imperative for the GEF. As a gender-re- 
sponsive approach is applied throughout the GEF project cycle, it 
also applies to evaluations, as clearly stated in the 2017 GEF Policy 
on Gender Equality. Evaluations must assess whether and how men 
and women are affected by changes to natural resource use and deci- 
sion making resulting from GEF outcomes. Wherever feasible, eval- 
uations should provide sex-disaggregated and gender-sensitive data. 
Units commissioning evaluations should strive for gender balance in 
the composition of evaluation teams. 

i Competencies and capacities. GEF evaluations require a range of 
expertise that may be technical, environmental, cultural, or within a 
social science or the evaluation profession. Units commissioning evalu- 
ations are responsible for selecting evaluators with sufficient experience 
and skills in the appropriate field/s, and for adopting a rigorous meth- 
odology for the assessment of results and performance. Evaluations of 
GEF activities shall make the best possible use of local expertise, both 
technical and evaluative. 
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2. Criteria 


25 


26 


Evaluations in the GEF explore four major criteria, in line with recent 


OECD-DAC guidance: 


a Relevance—the extent to which the intervention design and intended 
results were consistent with local and national environmental priorities 
and policies and to the GEF’s strategic priorities and objectives, and 
remained suited to the conditions of the context, over time. 

b Effectiveness—the extent to which the intervention achieved, or 
expects to achieve, results (outputs, outcomes and impacts, including 
global environmental benefits) taking into account the key factors 
influencing the results. 

c Efficiency—the extent to which the intervention achieved value for 
resources, by converting inputs (funds, personnel, expertise, equip- 
ment, etc.) to results in the timeliest and least costly way possible, 
compared to alternatives. 

d Sustainability—The continuation/likely continuation of positive 
effects from the intervention after it has come to an end, and its poten- 
tial for scale-up and/or replication; interventions need to be environ- 
mentally as well as institutionally, financially, politically, culturally 
and socially sustainable. 


Other criteria may be considered on a case by case basis, depending on 
the specific evaluation as well as new developments in the international 
good practice standards for evaluation.’ 


3. Indicators 


27 


In the context of the results frameworks (and related core corporate indi- 
cators) introduced in each replenishment of the GEF, projects and pro- 
grams shall adopt monitoring and evaluation systems with indicators that 
ensure evaluability. Indicators should be SMART—specific, measurable, 
attributable, relevant, and time-bound. These indicators will be reported 
in the terminal evaluations to assess performance against objectives and 
should have the following characteristics: 


a Specific. The indicator measures only the design element (output, 
outcome, or impact) that it is intended to measure and captures the 
essence of the desired result by clearly and directly relating to the 
achievement of an objective, and only that objective. 

b Measurable. There are practical ways to quantitatively and/or quali- 
tatively measure the indicator, i.e. the indicator has the capacity to be 
counted, observed, analyzed, tested, verified, or challenged. 

c Attributable. The indicators identify what changes occurred or are 
anticipated as a result of the intervention and whether the results are 
realistic. Attribution requires that changes in the targeted environ- 
mental and developmental issues can be linked to the intervention. 


28 
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d Relevant. The indicators establish levels of performance that are 
likely to be achieved in a practical manner, that reflect the expecta- 
tions of stakeholders, and are plausibly associated with the activities, or 
the indicator is valid in describing the underlying issues and processes 
followed to tackle those issues. 

e Time-bound, timely, trackable, and targeted. The indicators 
track progress in a cost-effective manner at appropriate intervals for a 
set period, with clear identification of the stakeholder group(s) to be 
affected by the intervention. 


Results in the GEF are measured by global environmental benefit indica- 
tors, according to the results frameworks approved in each replenishment 
phase. Social and economic co-benefits achieved while contributing to 
global environmental benefits are also measured. As per the GEF Policy 
on Gender Equality, the collection of sex-disaggregated data and infor- 
mation on gender, and the use of gender-sensitive indicators, sex-disag- 
gregated targets and results, as relevant, are to be regularly incorporated 
in monitoring and evaluation. Wherever possible, the geographic coor- 
dinates of project sites should be collected and used in monitoring and 
evaluation. 


4. Minimum requirements 


29 


30 


31 


Four minimum requirements must be applied to evaluation at 
the project and program levels (Boxes A1.1-A1.4). Minimum 
Requirements 1 and 2 must also be applied to monitoring, as confirmed 
by the GEF Monitoring Policy. The objectives and intended results of 
GEF-financed activities should be specific and measurable, so as to make 
it possible to monitor and evaluate the project and program effectively. 
Baseline data should be developed for the key results indicators. Agencies 
should ensure timely monitoring and evaluation planning at the project 
preparation stage. 

Terminal evaluations of programs and full- and medium-sized 
projects are to be conducted according to Minimum Requirement 
3. Once in full compliance with the provisions contained in Minimum 
Requirement 3, Agencies are expected to apply their internal arrange- 
ments to the conduct of terminal evaluations and their cost to ensure that 
evaluation reports of GEF-financed activities conform to GEF evaluation 
principles. Evaluations should provide lessons learned and recommenda- 
tions for future projects, programs, or policies. 

Terminal evaluations will assess results (outputs, outcomes, and 
impacts) according to the criteria of relevance, effectiveness, 
efficiency (cost-effectiveness), and sustainability, as applicable. 
Terminal evaluations will also assess GEF additionality, defined as the 
additional outcome (both environmental and otherwise) that can be 
directly associated with the GEF-supported project or program. To do so, 
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Box A1.1 


Minimum Requirement 1: design of monitoring and evaluation plans 


All projects must include a concrete and fully budgeted Monitoring and Evaluation 
Plan by the time of CEO endorsement for full-sized projects and CEO approval 
for medium-sized projects. Program Monitoring and Evaluation Plans describing 
the intended approach to monitoring and evaluation across the program, program 
rationale, the theory of change, results frameworks and indicators, and ways to 
ensure coherence across the child projects, must be included at program framework 
document (PFD) approval. Concrete and fully budgeted Program Monitoring and 
Evaluation Plans must be further detailed in the child project which supports the 
coordination, knowledge sharing, and monitoring and evaluation activities of the 
program, where applicable. 

Logical frameworks and/or theories of change should align, where appropriate, 
to the GEF’s results frameworks. Program Monitoring and Evaluation Plans must 
ensure coherence between program and child project objectives, indicators, and 
outcomes. Monitoring and evaluation Plans build in the possibility to adapt to 
changing conditions, if needed. Project and Program Monitoring and Evaluation 
Plans should contain the following: 


e SMART indicators for results and implementation linked appropriately to 
the GEF results frameworks, and including the following: 


e Applicable GEF indicators on global environmental benefits identified at 
each replenishment cycle 


e Socioeconomic co-benefits and sex-disaggregated/gender-sensitive 
indicators (where relevant) 


e Project site geographic coordinates (where feasible and appropriate) 


e Additional process and/or performance indicators that can deliver relia- 
ble and valid information to management 

e Project and program baselines, with a description of the problem to be 
addressed and relevant indicators 

e Periodic implementation reports, midterm reviews, and terminal evaluations 

e Organizational set-up and budgets for both monitoring and evaluation, 
where the budget for evaluation should be explicit and distinguished from 
monitoring activities 


32 


Agencies will apply the evaluative approach for assessing GEF’s addition- 


ality approved by the GEF Council in December 2018." 


Terminal evaluations of programmatic approaches, which are by 
nature designed as a set of coherent and synergistic interventions 
to achieve broader and longer-term results, must also assess the 
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Box A1.2 


Minimum Requirement 2: application of monitoring 
and evaluation plans 


Project and program monitoring will include implementation of the Monitoring 
and Evaluation Plan, comprising the following: 


e The identified indicators are actively measured, or if not, a reasonable expla- 
nation is provided 

° The baseline for the project or program is fully established and data are com- 
piled to review progress, and evaluations are undertaken as planned 

e The organizational set-up for monitoring and evaluation is operational, and 
its budget is spent as planned 


Box A1.3 


Minimum Requirement 3: terminal evaluation 
of GEF-financed activities 


All full- and medium-sized projects and all programs will need to be evaluated 
at the end of implementation. The evaluation will have the following minimum 
requirements: 


° The evaluation will be undertaken independent of project management, or 
if undertaken by project management, will be reviewed by the evaluation 
unit of the GEF Agency or by independent quality assurance mechanisms of 
the Agency 

¢ The evaluation will apply the international standards and minimum require- 
ments set forth in this Evaluation Policy 

e The evaluation will assess at a minimum: 

e Achievement of outputs and outcomes, and provide ratings for targeted 
objectives and outcomes, for projects. For programs, aggregated results will 
be reported. 

e Likelihood of sustainability of outcomes at termination for projects and the 
overall program 

e Whether Minimum Requirements 1 and 2 were met 

e An assessment of GEF additionality 

e An assessment of whether and how men and women are affected differently 
by changes to natural resource use and decision making resulting from GEF 
outcomes 


In addition, for programs, the terminal evaluation will provide an assessment of the 


coherence between program and child project theories of change and objectives, 


158 Appendix 1 


indicators, and programmatic results achieved; these results must demonstrate the 
program’s added value over comparable non-programmatic alternatives. 


e The terminal evaluation report will contain at a minimum: 

e Basic data on the evaluation, indicating when it took place, who was involved, 
its key questions, its methodology, including application of the four evalua- 
tion criteria (set forth in Subsection 2, above) 

e Basic data on the project or program, including actual GEF and other 
expenditures 

e Lessons for broader applicability 

e Evaluation terms of reference (in an annex) 

e The terminal evaluation report will be sent to the IEO immediately when 
ready, and at the latest, within 12 months of completion of project or program 
implementation 
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added value of implementing interventions as a program rather 
than as stand-alone projects or other comparable alternatives. 
Results must be measured according to Minimum Requirement 3. 

The Monitoring and Evaluation Plan for programs shall, at PFD approval, 
include arrangements for program-level as well as child project—level 
terminal evaluation. Agencies participating in the program will conduct 


Box A1.4 


Minimum Requirement 4: engagement of operational focal points 


Except for those GEF-financed activities for which OFP letters of endorsement 
are not required, all other projects and programs will engage the Operational 
Focal Points (OFPs) in monitoring and evaluation related activities. The following 
requirements shall be met: 


e The monitoring and evaluation plan will specify how the project or program 
will keep the relevant OFP informed and, where applicable and feasible, 
involved, while respecting the independent nature of evaluation 

e During implementation, the Agencies will inform the OFPs on monitoring 
and evaluation activities in the projects and programs that belong to their 
national portfolio 

e The OFPs will be informed of midterm reviews and terminal evaluations and 
will, where applicable and feasible, be briefed and debriefed at the start and at 
the end of evaluation missions. They will receive a draft report for comment, 
will be invited to contribute to the management response (where applicable), 
and will receive the final evaluation report within 12 months of project or 
program completion 

e The GEF Agencies will track application of the conditions specified here in 
their GEF-financed projects and programs. 
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III. 
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the terminal evaluations of the child projects they have implemented 
within the program. The Lead Agency will be responsible for submit- 
ting the terminal evaluation for the program, consistent with Minimum 
Requirement 3. 

The evaluation plan of jointly implemented projects must include 
the evaluation arrangements agreed upon by the Agencies at 
CEO endorsement. Partners’ responsibilities in ensuring evaluation of 
jointly implemented projects need to be discussed and agreed upon at the 
time of preparation to ensure cost-effectiveness, synergy, and avoidance 
of duplication in evaluation reporting. It is the responsibility of the Lead 
Agency of the jointly implemented project to guide this discussion and 
reach an agreement with all the other participating Agencies at design. 
The IEO follows the World Bank Policy on Access to Information." 
The GEF Policy on Access to Information states that “The Ethics 
Committee, the Ethics Officer, the Independent Evaluation Office, the 
Secretariat, and the Trustee are subject to the World Bank Policy on 
Access to Information.”!” 

The IEO shall be provided access by the Agency to the docu- 
mentation and information it needs to conduct its evaluations 
of GEF-financed activities. Staff of the IEO shall have access to the 
Agencies’ official records to enable their work as evaluators, in accordance 
with applicable Agency policies and rules governing such access. IEO 
staff shall also have access to Agency staff during evaluations. Relevant 
Agency representatives shall facilitate visits by IEO staff to project sites 
and meetings with stakeholders and government representatives. 
Evaluations shall be disseminated in accordance with widely 
accepted international standards, by establishing effective feedback 
loops to policy makers, operational staff, beneficiaries, and the general 
public. The disclosure of evaluation reports shall be ensured through 
posting on websites and dissemination of findings through knowledge 
products and events. GEF-related evaluation reports should be broadly 
and freely shared, and findings and lessons learned made available to pro- 
ject management. Evaluation reports should provide transparent infor- 
mation on data sets, sources, methodologies, and approach. 


Roles and responsibilities 


Different partners and stakeholders within the GEF have different roles 
and responsibilities with respect to monitoring and evaluation. The GEF 
Council provides the overall framework for enabling monitoring and 
evaluation, starting with an agreement on the overall objectives and the 
corporate and focal area results frameworks. The GEF Secretariat devel- 
ops and submits the plans and framework for monitoring progress against 
these objectives to the Council, and the IEO presents the overall evalu- 
ation work plan to report on the overall performance and effectiveness 
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of the GEF, to the Council. The GEF Council approves the IEO work 
program and budget, receives the evaluation reports, and decides on the 
management actions to respond to the evaluation recommendations. The 
IEO prepares a Semi-Annual Evaluation Report every six months and 
a comprehensive evaluation of the GEF every four years. Based on this 
information, the Council makes strategic and policy-level decisions. The 
GEF Agencies and their partners execute project, program, and portfo- 
lio monitoring and evaluation plans. The GEF Scientific and Technical 
Advisory Panel (STAP) provides advice on indicators and targets at the 
stage when policies, strategies, programs, and projects are prepared, as 
well as evaluation approaches in response to specific requests by the IEO. 
Figure A1.2 and Table A1.1 provide the broad framework of the main 
roles and responsibilities of the key partners for monitoring and evalua- 
tion in the GEF. 

Each GEF Agency has its own system of governance and rules 
and regulations governing the implementation of activities, as 
well as the evaluation of these activities. The GEF Council can 
adopt principles, norms, and standards for those parts of the GEF for 
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Table A1.1 Key roles and responsibilities of GEF partners in evaluation 


Partner Key monitoring and evaluation roles and responsibilities 
Council e Develop the overall policy on monitoring and evaluation 
e Provide an enabling environment for monitoring and 
evaluation 
e Oversee the evaluation function and guarantee IEO 
independence 
e Receive independent evaluation reports and decide on fol- 
low-up actions to implement evaluation recommendations 
IEO e Conduct independent GEF evaluation 
e Validate terminal evaluations prepared by Agencies 
e Undertake post completion evaluation for a sample of projects 
e Assess the quality of project and program evaluations 
e Set minimum requirements for evaluation 
e Prepare the Management Action Record 
e Share and disseminate evaluative knowledge 
Secretariat e Set results frameworks at focal area and corporate levels 
e Monitor the GEF portfolio across Agencies and focal areas 
e Report on and incorporate lessons from portfolio monitoring 
e Review monitoring and evaluation requirements in project 
and program proposals 
GEF Agency - e Monitor the Agency GEF portfolio 


operational units 


GEF Agency 
- evaluation units 


STAP 


GEF OFPs 


Other stakeholders 
(CSOs, private sector, 
communities) 


e Report Agency project, program, and portfolio progress, 
results, and learning 

e Ensure monitoring at the project and program levels, as 
appropriate 

e Manage project and program implementation adaptively 

e Systematically involve national partners and share project 
monitoring and evaluation information at the national level 


e Conduct and/or validate terminal evaluations of projects 
and programs 

e Conduct corporate Agency evaluations 

e Mainstream the GEF into relevant Agency evaluations 


e Advise on scientific/technical matters in monitoring and 
evaluation 
e Provide support on scientific and technical indicators 


e Collaborate on monitoring and evaluation at project, pro- 
gram, and portfolio levels 


e Participate in monitoring activities 
e Provide views and perceptions to evaluations 


which it is directly responsible, such as the GEF Secretariat, the IEO, 
and the STAP. The Council can also require minimum standards and 
minimum procedures to be applied to the evaluation of the activities that 
it funds. Specific roles and responsibilities for evaluation are detailed for 
each partner in the following subsections. Specific roles and responsibili- 
ties for monitoring are detailed in the GEF Monitoring Policy. 
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40 The GEF Council ensures accountability and oversight of GEF 


41 


performance and results. The Council develops, adopts, and over- 
sees the operational policies and programs for GEF-financed activities; 
reviews the operations of the GEF with respect to its purposes, scope, 
and objectives; and ensures that the GEF policies and work program, 
including operational strategies, programs, and projects, are moni- 
tored and evaluated on a regular basis. The Council also establishes 
the GEF Evaluation Policy for the GEF and approves the IEO’s work 
program and budget. The Council receives the independent evalua- 
tions conducted by IEO, including the comprehensive evaluation of 
the GEF, and decides on follow up actions from evaluation recom- 
mendations. The Council uses independent evaluation to comple- 
ment a larger system of financial oversight and accountability within 
the GEF Trustee and Agencies. On behalf of the Council, the GEF 
Trustee ensures the maintenance of appropriate records and accounts 
of the GEF trust fund and provides for their audit, in accordance with 
the rules of the Trustee. 

The GEF Council provides an enabling environment for eval- 
uation activities in line with internationally accepted standards 
and guarantees the independence of the IEO. The Council ensures 
that adequate resources are allocated to enable the independent evalua- 
tion function to operate effectively and with independence and that eval- 
uators have the freedom to conduct their work without repercussions for 
career development. It also appoints a professionally competent director 
to lead the IEO. The Council promotes transparency, participation, and 
disclosure of evaluation findings, and ensures that sufficient time is ded- 
icated to discussion of evaluation issues at Council meetings. The GEF 
Council, together with the GEF CEO and the IEO Director, is respon- 
sible for the use of evaluation products including the systematic consid- 
eration of findings, conclusions, and recommendations, and lessons, for 
decision making on GEF programs and policies. 


2. Independent evaluation office of the GEF 


42 The IEO has the central role of ensuring the independent eval- 


uation function within the GEF. The IEO operates as an organiza- 
tional unit independent of GEF Secretariat management or Agency. The 
IEO prepares the overall evaluation policy for the GEF to be endorsed 
by the Council; sets the minimum requirements for evaluation in the 
GEF partnership, sets an evaluation agenda for approval by the Council, 
validates the terminal evaluations of projects and programs by Agencies, 
conducts a broad spectrum of independent evaluations and shares evalu- 
ative evidence within and outside the GEF. 
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45 
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The IEO pursues the goals of improved accountability and learning 
through two main functions: 


a An evaluative function. The main function of the IEO is to inde- 
pendently evaluate the effectiveness of the GEF at the project, pro- 
gram, portfolio, and institutional levels. 

b A normative function. The IEO is tasked to set minimum evalua- 
tion requirements, and evaluation standards within the GEF to ensure 
improved and consistent measurement of GEF results. 


The IEO is independent from both the policy-making process 
and the delivery and management of assistance. This guarantees 
that data gathering and analysis and judgments on criteria, findings, and 
recommendations will not be influenced by conflicts of interest or undue 
interference by management at any level. The IEO requests feedback 
and comments on draft reports from all stakeholders consulted during 
the evaluation, ensuring due diligence and verification of any eventual 
evidence gaps. The Secretariat, Agencies, and other affected parties may 
receive, comment, and respond to draft and final evaluation reports, but 
cannot approve, hold back, request changes, or otherwise modify such 
draft and final evaluation reports. IEO evaluation reports are submitted 
directly and simultaneously to the GEF Council and the GEF Secretariat. 
The independence of the IEO evaluation function is overseen 
by the GEF Council. The GEF Instrument, amended at the Fifth 
GEF Assembly in May 2014, states that “there shall be an independent 
evaluation office headed by a director, appointed by and reporting to 
the Council, whose responsibility it is to carry out independent evalua- 
tions consistent with decisions of the Council.”!* The term of the IEO 
Director will be determined by the Council. A Council Selection and 
Review Committee is formed to oversee the processes for appointing 
the Director and for conducting his/her performance objective reviews. 
The Director cannot join the GEF in any other capacity after completion 
of his/her mandate with the GEF. The Director is directly accountable 
to the GEF Council for the work of the IEO and may propose to the 
Council any measure he/she believes is necessary to ensure evaluation 
independence. 

The Director manages the IEO and its budget by implementing strategic 
decisions by the GEF Council, providing overall direction and resource 
management, and strengthening institutional relationships. The Director 
is solely responsible for personnel decisions in the IEO in accordance 
with staff rules. In the exercise of these functions, the IEO Director 
participates in the GEF Council, the Assembly, and in the replenishment 
meetings on evaluation issues, and responds to Council requests on any 
related matters. The Council has direct access to the Director and his/her 
staff, and the Director may communicate directly with Council mem- 
bers during and between Council meetings or arrange special meetings 


164 Appendix 1 


47 


48 


49 


50 


as deemed appropriate and without prior clearance from anyone outside 
the IEO. Furthermore, the Director may propose decisions to the GEF 
Council on a no-objection basis between Council sessions. 

To avoid conflict of interest, the Director establishes clear con- 
flict-of-interest rules for the IEO staff. In this connection, an 
evaluation will not be entrusted to an IEO staff member who has been 
responsible in the past for the design, implementation, or supervision of 
the project, program, portfolio, strategy, or policy to be evaluated. The 
IEO does not engage consultants who have worked previously either as 
individuals or through private consulting firms and/or nonprofit organi- 
zations on the design or implementation of a project, program, portfolio, 
strategy, or policy to conduct evaluation analysis or prepare evaluation 
reports of the same. 

Under the Director’s leadership, the IEO has the responsibility 
for undertaking independent evaluations that involve a set of 
projects from more than one Agency. These evaluations are typi- 
cally on a strategic level, on focal areas or programs, or on cross-cutting 
themes. Institutional evaluations are also undertaken. In addition, the 
IEO validates project terminal evaluations for those Agencies in which 
the evaluation function is not fully independent. It does so according 
specific evaluation guidelines.'* Where possible and to prevent dupli- 
cation and to promote synergies, the IEO collaborates with evaluation 
units of the GEF Agencies. Within the GEF, the IEO facilitates cooper- 
ation with and among the GEF partners on matters of evaluation. This 
includes the establishment of procedures and guidelines on evaluation 
based on the internationally recognized good practice standards. 

In support of the Council’s oversight role and to promote 
accountability, the Director of the IEO reports directly and 
regularly to the Council on evaluation findings. The evaluative 
evidence is presented in Semi-Annual Evaluation Reports (SAER) 
which summarize the data and analyses, findings, conclusions, and rec- 
ommendations of the individual evaluations completed in the semes- 
ter being reported on. The SAER is presented as a working document 
and includes proposed Council decisions based on the recommendations 
from the evaluations. The evaluations themselves are submitted to the 
Council as information documents. 

The evidence presented in the IEO’s evaluations is either developed by 
the IEO itself or extracted and independently verified from evaluations 
by GEF Agency evaluation units. Data and information sources include 
monitoring data (once independently verified), the academic literature, 
primary data collected through interviews, field visits, remote- sensing, 
and other relevant credible sources. The IEO undertakes post comple- 
tion evaluation for a sample of projects. The IEO also reviews project 
terminal evaluation reports submitted by the Agencies. Terminal evalu- 
ation reports focus on the ex post results of GEF projects and programs. 
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The IEO Director prepares a four-year rolling work program 
and budget request and submits these directly to the Council 
for approval. The monitoring and evaluation budgetary needs of the 
Agencies and the GEF Secretariat are addressed separately in the GEF 
corporate budget and through project fees. The IEO four-year work pro- 
gram and budget is developed in consultation with the GEF partnership 
and is approved by the Council. It reflects a phased approach over a GEF 
replenishment period to ensure adequate evaluation coverage to pro- 
mote accountability and learning. For every major evaluation, the IEO 
prepares an approach paper which is shared for comment with all the 
partners involved before finalization to allow for stakeholder feedback 
on the evaluation design. 

The IEO ensures follow-up of evaluation recommendations 
through the management action record system, as part of its 
accountability function. A Management Action Record table con- 
taining all IEO evaluation recommendations, management responses 
and related Council decisions is compiled annually and circulated to the 
GEF Secretariat to rate and report progress on actions implemented on 
Council decisions. The IEO provides an independent assessment of the 
rate of adoption of Council decisions. 

The IEO supports knowledge sharing and establishes systems 
to disseminate lessons learned and documents best practices 
from evaluations and provides independent evaluative evidence 
to the GEF knowledge base. The IEO supports knowledge sharing 
by ensuring the highest standards in accessibility and presentation for its 
published reports. The IEO develops learning products based on eval- 
uations, disseminates findings through the IEO website, publications, a 
spectrum of conferences, communities of practice, web platforms such as 
Earth-Eval, social media, to share evaluation findings within and outside 
the GEF partnership. 

The IEO establishes appropriate quality assurance mechanisms 
for its major evaluations and adopts the highest standards recognized in 
the international evaluation community. These quality assurance mech- 
anisms address evaluation approaches and methods, data gathering and 
analysis, and include reporting on evaluation findings and conclusions. 
The IEO works in close partnership with the global evaluation 
community. The IEO remains on the cutting edge of emerging and 
innovative methodologies in environmental evaluation. It consults and 
collaborates with all relevant partners to foster a network of evaluation 
professionals who may add value to GEF operations and results. 


3. GEF Secretariat 


56 


The GEF Secretariat is responsible for monitoring and reporting 
on the results of the overall GEF Portfolio, in accordance with 
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the results frameworks set by the Council in each replenishment 
period. The Secretariat reviews all projects and programs prior to their 
approval to ensure they meet GEF monitoring and evaluation require- 
ments, including the use of indicators and targets to ensure alignment 
with focal area objectives. 

The GEF Secretariat ensures that findings and recommenda- 
tions from evaluations are followed up on with respect to GEF 
policies, programs, and procedures, and that related Council 
decisions are implemented. The Secretariat ensures that results 
and lessons are adequately reflected in public information about the 
GEF. This includes activities to gather and disseminate best practices 
to improve portfolio quality. In support of evaluation, the Secretariat 
responds promptly and fully to all IEO requests for information relating 
to GEF projects, programs, and policies; and coordinates the GEF man- 
agement response to IEO evaluations. 


4. GEF Agencies 


Operational Units 
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In line with the Minimum Fiduciary Standards for GEF Partner 
Agencies,” the Agencies are responsible for ensuring that pro- 
jects and programs are properly designed with monitoring 
and evaluation plans, and that projects are adequately moni- 
tored during implementation. The GEF Agencies are responsible for 
developing monitoring plans with appropriate performance and results 
indicators for projects and programs; for adequately monitoring pro- 
ject and program activities, production of outputs, and progress toward 
outcomes. When designated as a program’s Lead Agency, an Agency is 
responsible for monitoring the program as well as the child projects it 
directly implements. The Agencies implementing the other child pro- 
jects in the program are responsible for their monitoring. To ensure that 
results can be analyzed across Agencies in a consistent manner, project 
logical frameworks and/or theories of change should be aligned with the 
GEF focal area results frameworks, as applicable. 

Agencies must undertake midterm reviews for programs and 
full-sized projects under implementation for adaptive manage- 
ment purposes. Midterm reviews are also encouraged for medium-sized 
projects and enabling activities where appropriate and feasible. These 
reports are submitted to the GEF Secretariat as part of annual reporting. 
The Agencies support the IEO by responding promptly and 
fully to requests for information or support relating to evalu- 
ation of GEF activities, and by making project and program 
documentation available to the IEO. As per the updated GEF Policy 
on Cofinancing, Agencies provide information on the actual amounts, 
sources, and types of cofinancing and investment mobilized in their 
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midterm reviews and terminal evaluations. The Agencies encourage 
OFPs to be fully informed of and consulted on the conduct of terminal 
evaluations, and that they receive and comment on terminal evaluations. 


Evaluation Units 


61 
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Agencies ensure the conduct of required terminal evaluations 
of GEF-supported projects and programs in their portfolio. 
Depending on the Agency’s institutional structure as well as internal 
rules and procedures, two scenarios exist: (1) the evaluation unit con- 
ducts project terminal evaluations, and (2) the evaluation unit validates 
the evaluations managed by operational units. Agencies are responsible 
for the terminal evaluation of the child projects they directly implement 
in a program. In addition, the Lead Agency will be responsible for sub- 
mitting the terminal evaluation for the program. Consistent with the 
GEF Project and Program Cycle Policy, any project and program evalu- 
ations must be shared with the IEO. 

GEF Agency Evaluation Units may be called upon to explore with the 
IEO possible areas of common interest and cooperation and opportuni- 
ties for joint evaluations. For evaluations covering issues of GEF concern 
and the GEF portfolio, the evaluation units engage with the IEO on the 
terms of reference, approach, and scope. 

The Agency evaluation units coordinate with the IEO on norms, stand- 
ards, and quality of evaluations when it comes to GEF-financed activ- 
ities. Agencies are expected to provide adequate financial support for 
their evaluation units to undertake their work in a way that does not 
compromise the independent conduct of evaluations. 


5. Scientific and Technical Advisory Panel 
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Upon receipt of specific requests from the IEO, the STAP may contribute 
scientific and technical advice, data or other information that may be use- 
ful to evaluation. Such requests may pertain to opinions on the evalua- 
bility of scientific aspects and related methodologies for measuring global 
environmental impacts in response to evaluation approach papers, terms 
of reference, or reports. STAP members may also be requested for direct 
support to an evaluation while respecting the independence of the IEO. 


6. GEF Operational Focal Points 
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Several entities in GEF member countries are involved in evaluation in 
different ways. Many systems on local and global environmental benefits. 
These initiatives may include improving basic census data, establishing 
national and project baselines, establishing participatory environment 
and natural resource monitoring schemes, using national communica- 
tions and inventories of global environmental benefits, among others. 
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In line with GEF operational principles on country ownership, 
evaluation activities will be consultative. The GEF OFPs will be 
fully consulted with and informed by the GEF Agencies and the IEO on 
the planning, conduct, and results of any evaluation activity performed 
in their country, and they in turn will respect the independence of the 
evaluation. 

The GEF OFPs play a key role in facilitating access to staff mem- 
bers of government institutions involved in GEF projects during 
evaluations. They may promote the use of, follow-up to, and action 
on evaluation recommendations related to GEF matters and directed at 
the regional, national, and project levels. They also play an important 
role in keeping national stakeholders (including the civil society organ- 
izations involved in GEF activities) fully consulted with, informed on, 
and involved in the plans, conduct, and results of country-related GEF 
evaluation activities. 


7. Other stakeholders 
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A considerable number of locally and internationally based stakehold- 
ers are involved in GEF evaluation activities. These stakeholders are the 
individuals, groups, or institutions that have an interest or stake in the 
outcome of a GEF-financed project or program, including those poten- 
tially affected by a project or program. Stakeholders may include national 
project or program executing agencies; groups contracted to conduct 
activities at various stages of the project or program; and other civil soci- 
ety groups including local community members who may have an inter- 
est in the project or program, or who are living in the project or program 
area, or who are dependent for part of their livelihoods or in times of 
stress on the natural resources of the project or program area. Their 
involvement in evaluation depends on the project or program and their 
role. For example, academic institutions or private sector companies may 
support evaluation activities directly and provide outside perspectives 
and expertise. CSOs may play an important role in providing feedback 
as beneficiaries or as representatives of community groups. 

Consistent with provisions in the GEF Instrument and with the 
GEF Stakeholder Engagement Policy, there shall be transpar- 
ency in the preparation, conduct, reporting, and evaluation of 
public involvement activities in all projects and programs. This 
includes full disclosure of all nonconfidential information, and consul- 
tation with major groups and representatives of local communities. GEF 
evaluations should involve project stakeholders, both as participants and 
contributors and as users and beneficiaries as appropriate. Local stake- 
holder participation and participatory approaches in evaluation are par- 
ticularly necessary in projects and programs that affect the incomes and 
livelihoods of local groups, especially disadvantaged populations in and 
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around project sites (for example, indigenous and other local communi- 
ties, women, and poor households). 
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OPS6 Independent Advisory 
Panel Statement 


Introduction 


The Independent Advisory Panel prepared this Statement after reviewing 
the final draft of OPS6. The Panel acknowledges the impressive work that 
has been done by GEF’s Independent Evaluation Office (IEO) to produce 
the OPS6 report, which takes into account the complexities that the GEF 
faces and synthesizes the enormous amount of information collected through 
29 independent evaluations. These detailed reports were building blocks for 
the OPS6, which brought rigor and depth to its findings and conclusions, 
distilling the essence from those evaluations. 

This Statement discusses the process followed for OPS6 and focuses on 
the OPS6 report, both on the quality of the evidence and methods used as 
well as on the quality of the arguments, considering first the links between 
evidence and conclusions and second the links between conclusions and 
recommendations. Finally, the Panel provides comments on a set of key 
issues. 


On the OPS6 process 


The evaluation briefs prepared by IEO of completed and ongoing evaluations 
summarized in a four-page format, including information on the status of the 
various evaluations, preliminary findings and collected evidence, as well as the 
construction of a dedicated website for OPS6, provided an appropriate means 
of communicating in real-time the results of the evaluation. 

The Independent Advisory Panel reviewed and made observations on the 
approach paper for OPS6. It also provided comments and suggestions on an 
annotated outline of the report at a face-to-face meeting with the IEO. The 
Panel reviewed and provided comments on a zero draft of the report, which 
was prepared and circulated 6 weeks after that meeting. Finally, as mentioned 
in the Introduction, the Independent Advisory Panel prepared this statement 
after reviewing the final draft of OPS6, which to a great extent incorporated 
the comments of the Panel on the zero draft. 
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On the OPS6 report 


Quality of the evidence and methods 


The Panel commends the use of multiple sources of evidence and the applica- 
tion of different methods, thus allowing for adequate triangulation to ensure 
the reliability and veracity of the findings. However, a more detailed pres- 
entation of the methods used would have been worthwhile. 


Quality of the arguments 


The conclusions are supported by the evidence, although this is not evident 
at a glance in all cases. The recommendations provide valuable guidance on 
how to deal with the issues identified in the report. 


Some key issues 


This section addresses some issues that the Panel feels are not fully considered 
in the conclusions or presented in the recommendations. 


1 The GEF project data base is not yet comprehensive, consistent, updated 
and accessible. This information is a global public good and its consoli- 
dation and appropriate dissemination would make an important contri- 
bution if properly curated and placed in the public domain. 

2 The fundamental systemic and urgent nature of the issues at stake should 
be acknowledged, moving beyond a largely project-driven logic in the 
funding cycle. 

3 Although, as indicated in conclusion 4 of the OPS6 report, it is too early 
to assess the performance of the Integrated Approach Pilots, the Panel 
believes that the GEF should continue pursuing an integrated approach 
and that it would be appropriate to include a full assessment of the Pilots’ 
performance in OPS7. 

4 Although the quality of private sector engagement is improving there is 
still a need for greater clarity on how the GEF sees the role of the private 
sector and vice versa. Specifically, whether it is seen by the GEF pri- 
marily as a mechanism for securing additional funding, as a mechanism 
to achieving desired environmental outcomes or both, and whether the 
private sector sees the GEF as primarily creating the enabling environ- 
ment for investment. To enhance the engagement strategy, there should 
be greater understanding and recognition that the private sector is not a 
single entity but a complex mosaic of for-profit businesses, including the 
financial sector. 

5 Even though the report states that the GEF appears to have a greater risk 
appetite and tolerance than other financiers, the very high percent of 
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completed GEF-4 projects, which had outcomes rated in the satisfactory 
range (85%, exceeding the 75% performance target set out in the replen- 
ishment), may be a sign of a rather risk averse, insufficiently innovative 
project portfolio. The Panel considers that the GEF is well placed to take 
more risks and play a more innovative and transformative role. 


The Panel would also like to emphasize the importance and urgency of 
implementing a foreign exchange risk management mechanism, as indicated 
in the second part of recommendation 4. 

Finally, the Panel commends the GEF for its excellent responsiveness to the 
Conventions, as reported in OPS6. 


Overall assessment 


The Independent Advisory Panel considers that the Sixth Comprehensive 
Evaluation of the GEF provides solid evaluative evidence to inform the nego- 
tiations for the seventh replenishment of the GEF and therefore OPS6 fulfills 
its purpose. 


Appendix 3 
Selected global environment 
facility evaluations and studies 


An Evaluative Approach to Assessing GEF’s Additionality, 2018 
www.thegef.org/council-meeting-documents/evaluative-approach- 
assessing-gef-s-additionality 

Annual Performance Report, 2014 

www.gefieo.org/evaluations/apr-2014 


Annual Performance Report, 2018 
www.thegef.org/sites/default/files/council-meeting-documents/ 
EN_GE.ME_C.54.Inf_.02_Annual_Performance_Report_2017_0.pdf 
Biodiversity Focal Area Study, 2017 
www.gefieo.org/evaluations/biodiversity-study-2017 

Climate Change Mitigation: GEF Support to Market Change in China, India, 
Mexico, and Russia, 2014 

www.gefieo.org/evaluations/ccm-2014 

Comparative Advantage, Adequacy of Funding/Financing, Health of the 
Expanded GEF Partnership and Governance Structure, 2017 
www.gefieo.org/evaluations/gef-partnership-governance-2017 

Evaluation of GEF Engagement with the Private Sector, 2017 
www.gefieo.org/evaluations/private-sector-2017 

Evaluation of GEF Support for Transformational Change, 2017 
www.gefieo.org/evaluations/transformational-change 

Evaluation of GEF Support to Scaling up Impact, 2019 
www.gefieo.org/evaluations/scaling 

Evaluation of GEF’s Support to Mainstreaming Biodiversity, 2019 
www.gefieo.org/evaluations/biodiversity-mainstreaming-2018 

Evaluation of Incremental Cost Assessment, 2007 
www.gefieo.org/evaluations/incremental-cost-assessment-2007 


Evaluation of Programmatic Approaches in the GEF, 2018 
www.gefieo.org/evaluations/programmatic-approaches-2016 
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Evaluation of the GEF-UNIDO Global Cleantech Innovation Programme, 
2018 
www.gefieo.org/evaluations/cleantech-programme-2018 


Formative Review of the Integrated Approach Pilot Programs, 2018 
www.gefieo.org/council-meetings/council-documents/gefmec53inf04 


The GEF in the South China Sea and Adjacent Areas, 2012 
www.gefieo.org/evaluations/scs-2012 


The GEF Response to the Crisis — What Can We Learn from Evaluation?, 
2020 
www.gefieo.org/documents/gef-response-crisis-what-can-we-learn- 
evaluation 

IEO Brief: A Value for Money Analysis of GEF Interventions in Land 
Degradation and Biodiversity, 2018 
www.gefieo.org/evaluations/synopsis/vfm-2016-land-degradation-brief 
Impact Evaluation of GEF Support to Protected Areas and Protected Area 
Systems, 2016 

www.gefieo.org/evaluations/impact-pa-support-2016 

Impact of GEF Support on National Environmental Laws and Policies in 
Selected Countries, 2017 
www.gefieo.org/evaluations/regulatory-reform-2017 

OPS2: The First Decade of the GEF: Second Overall Performance Study, 
2002 

www.gefieo.org/evaluations/ops2 


OPS3: Progressing Toward Environmental Results: Third Overall 
Performance Study of the Global Environment Facility, 2005 
www.gefieo.org/evaluations/ops3 


OPS4: Progress toward Impact: Fourth Overall Performance Study of the 
GEF, 2010 

www.gefieo.org/evaluations/ops4 

OPS5: At Crossroads for Higher Impact: Fifth Overall Performance Study of 
the GEF, 2014 

www.gefieo.org/evaluations/ops5-final-report 

OPS6: The GEFF in the Changing Environmental Finance Landscape: Sixth 
Comprehensive Evaluation of the GEF (OPS6), 2017 
www.gefieo.org/evaluations/ops6-report 


OPS6: Sixth Comprehensive Evaluation of the GEF (OPS6) Approach Paper, 
2016 
www.gefieo.org/council-meetings/council-documents/gefmec5007 


Program Evaluation of the Least Developed Countries Fund (LDCF), 2016 
www.gefieo.org/evaluations/Idcf-2016 
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Program Evaluation of the Special Climate Change Fund (SCCF), 2017 
www.gefieo.org/evaluations/sccf-2017 


The Role of Local Benefits in Global Environmental Programs, 2006 
www.gefieo.org/evaluations/lb-global-environmental-programs-2006 


Strategic Country Cluster Evaluation (SCCE): Small Island Developing States 
(SIDS), 2019 
www.gefieo.org/evaluations/scce-sids 


Strategic Country Cluster Evaluation (SCCE): Sahel and Sudan-Guinea 
Savanna Biomes, 2020 

www.gefieo.org/evaluations/scce-biomes 

A Study on the Health Co-benefits of GEF Chemicals and Waste Focal 
Area, 2020 
www.gefieo.org/sites/default/files/documents/co-benefits-cw-study- 
2019.pdf 

Value for Money Analysis for GEF Land Degradation Projects, 2018 
https://www.gefieo.org/evaluations/vfm-2016-land-degradation 

Value for Money Analysis of GEF Interventions in Support of Sustainable Forest 
Management, 2019 
www.gefieo.org/evaluations/vfm-2019-forest-management 
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Global environment facility 


projects referenced in this book 


Project or program name 


Alternatives to DDT Usage for 
the Production of Anti- 
fouling Paint 

Amazon Region Protected 
Areas Program (ARPA) 

Amazon Region Protected 
Areas Program Phase 2 

China Renewal Energy 
Scaling-Up Program 
(CRESP) Phase II 

China Utility-Based Energy 
Efficiency Finance Program 
(CHUEE) 

Cities-IAP: Sustainable Cities 
Integrated Approach Pilot 
(IAP-PROGRAM) 


Climate Change Adaptation 
Project, Phase I 

Comm-IAP: Taking 
Deforestation Out of 
Commodity Supply Chains 
(IAP-PROGRAM) 

Conservation and Sustainable 
Use of the Mesoamerican 
Barrier Reef 

Coral Reef Rehabilitation and 
Management Project 
(COREMAP I) 

Coral Reef Rehabilitation and 
Management Project Phase II 
(COREMAP II) 


GEF ID 
number 


2932 


771 


4085 


4493 


2624 


9077 


3243 


9072 


837 


116 


1829 


Implementing 
agency 


UNDP 


WB 
WB 


WB 


WB 


WB 


WB 


INDP 


WB 


WB 


WB 


Countries 


China 


Brazil 
Brazil 


China 


China 


Global, Brazil, Cote 
d'Ivoire, China, India, 
Mexico, Malaysia, Peru, 
Paraguay, Senegal, 
Vietnam, South Africa 

Philippines 


Global 


Regional, Belize, 
Guatemala, Honduras, 
Mexico 

Indonesia 


Indonesia 


Project or program name 


GEF ID 
number 


Implementing 
agency 
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Countries 


DBSB: Agricultural Pollution 
Control Project — under 
WB-GEF Strategic 
Partnership for Nutrient 
Reduction in the Danube 
River and Black Sea 

DBSB: Integrated Nutrient 
Pollution Control Project — 
Under the WB-GEF 
Investment Fund for Nutrient 
Reduction in the Danube 
River and Black Sea 

Demonstrating and Promoting 
Best Techniques and Practices 
for Reducing Health-care 
Waste to Avoid Environmental 
Releases of Dioxins and 
Mercury 

Demonstration of Alternatives 
to Chlordane and Mirex in 
Termite Control 

Demonstration project for 
Phasing-out and Elimination 
of PCBs and PCB-Containing 
Equipment 

Ecomarkets 

Enabling activities for the 
Stockholm Convention on 
Persistent Organic Pollutants 
(POPs): National 
Implementation Plan for 
Mauritius 

Enabling Activities to Facilitate 
Early Action on the 
Implementation of the 
Stockholm Convention on 
Persistent Organic Pollutants 
(POPS) in the Republic of 
Macedonia 

Food-IAP: Fostering 
Sustainability and Resilience 
for Food Security in Sub- 
Saharan Africa — An Integrated 
Approach (IAP-PROGRAM) 


1159 


2970 


1802 


2359 


2875 


671 
1824 


1518 


9070 


WB 


WB 


UNDP 


WB 


UNIDO 


WB 
UNDP 


UNIDO 


IFAD 


Romania 


Romania 


Global, Argentina, India, 
Lebanon, Latvia, 
Lebanon, the 
Philippines, Senegal, 


Tanzania, and Vietnam 


China 


North Macedonia 


Costa Rica 


Mauritius 


North Macedonia 


Regional, Burkina Faso, 
Burundi, Ethiopia, 
Ghana, Kenya, Malawi, 
Niger, Nigeria, Senegal, 
Eswatini, Tanzania, 
Uganda 
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GEF ID Implementing 

Project or program name number agency Countries 

Global Wildlife Program 10200 WB Global, Angola, Bhutan, 
Belize, Ecuador, 
Indonesia, India, 
Cambodia, Madagascar, 
Malaysia, Namibia, 
Nigeria, Panama, 
Pakistan, Chad, South 
Africa, Congo DR 

Hai River Basin Integrated 1323 WB China 
Water Resources Management 

High Efficiency Lighting Pilot 575 WB Mexico 

Improvement of DD'T-based 2629 UNDP China 
Production of Dicofol and 
Introduction of Alternative 
Technologies Including IPM 
for Leaf Mites Control in 
China 

IND-BD Mainstreaming 3936 UNDP India 
Coastal and Marine 
Biodiversity Conservation 
into Production Sectors in the 
Godavari River Estuary in 
Andhra Pradesh State 

IND-BD: GEF Coastal and 3661 UNDP India 
Marine Program (IGCMP) 

Institutional Capacity Building 101 WB Uganda 
for Protected Areas 
Management and Sustainable 
Use (ICB-PAMSU) 

LAC Regional Sustainable 2767 WB Regional, Argentina, 
Transport and Air Quality Brazil, Mexico 
Project 

Lighting the “Bottom of the 2950 WB Regional, Ghana, Kenya 
Pyramid” 

Mainstreaming Market-based 2884 WB Costa Rica 
Instruments for 
Environmental Management 
Project 

MENA - Desert Ecosystems 4620 WB Regional, Algeria, Egypt, 
and Livelihoods Program Jordan, Morocco 
(MENA-DELP) 

Namib Coast Biodiversity 1505 WB Namibia 
Conservation and 
Management (NACOMA) 

Namibian Coast Conservation 4669 WB Namibia 


and Management Project 


Project or program name 


GEF ID 
number 


Implementing 
agency 
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Countries 


PRC-GEF Partnership: An 
IEM Approach to the 
Conservation of Biodiversity 
in Dryland Ecosystems 
— Under the PRC-GEF 
Partnership on Land 
Degradation in Dryland 
Ecosystem Program 

PRC-GEF Partnership: 
Capacity and Management 
Support for Combating Land 
Degradation in Dryland 
Ecosystems 

PRC-GEF Partnership: 
Forestry and Ecological 
Restoration in Three 
Northwest Provinces 
(Formerly Silk Road 
Ecosystem Restoration 
Project) 

PRC-GEF Partnership: Land 
Degradation in Dryland 
Ecosystems: Project I 
— Capacity Building to 
Combat Land Degradation 

PRC-GEF Partnership: 
Mainstreaming Biodiversity 
Protection within the 
Production Landscapes and 
Protected Areas of the Lake 
Aibi Basin 

PRC-GEF Partnership: 
Sustainable Development in 
Poor Rural Areas 

Promoting Payments for 
Environmental Services (PES) 
and Related Sustainable 
Financing Schemes in the 
Danube Basin 

Protected Areas Management 
and Sustainable Use (PAMSU) 

PSG: Sustainable Land 
Management Project 2 

Reducing Transboundary 
Degradation in the Kura-Aras 
Basin 

Renewable Energy 
Development 


2369 


3484 


3483 


956 


3611 


3608 


2806 


1830 


5220 


1375 


446 


IFAD 


Asian 
Development 
Bank 


Asian 
Development 
Bank 


Asian 
Development 
Bank 


WB 


WB 


UNEP 


WB 
WB 


UNDP 


WB 


China 


China 


China 


China 


China 


China 


Regional, Bulgaria, 
Romania 


Uganda 
Ethiopia 


Regional, Armenia, 
Azerbaijan, Georgia 


China 


(Continued) 


180 Appendix 4 


Project or program name 


GEF ID 
number 


Implementing 
agency 


Countries 


Renewable Energy Scale Up 
Program (CRESP), Phase I 

Rio de Janeiro Integrated 
Ecosystem Management in 
Production Landscapes of the 
North-Northwestern 
Fluminense 

Rural Electrification and 
Renewable Energy 
Development 

Sanjian Plain Wetlands 
Protection Project 


Scaling up Risk Transfer 
Mechanisms for Climate 
Vulnerable Agriculture-based 
Communities in Mindanao 

SFM: Promotion of Sustainable 
Forest and Land Management 
in the Vietnam Uplands 

SIP: Country Program for 
Sustainable Land Management 
(ECPSLM) 

SP-SFIF: West Africa Regional 
Fisheries Program (WARFP) 


SPWA-BD: Participatory 
Biodiversity Conservation and 
Low Carbon Development in 
Pilot Ecovillages in Senegal 

Strengthening the Capacity of 
the Protected Area System to 
Address New Management 
Challenges 

Sustainable Land, Water, and 
Biodiversity Conservation and 
Management for Improved 
Livelihoods in Uttarakhand 
Watershed Sector Project 
(SLEM) 

Sustainable Management of 
POPs in Mauritius 

The Global Cleantech 
Innovation Programme for 
SMEs 

Uruguay Wind Energy 
Programme (UWEP) 

West Africa Regional Fisheries 
Program, Additional Financing 


943 


1544 


1209 


1126 


4967 


3627 


2794 


3558 


4080 


4729 


3471 


3205 


9811 


2826 


9360 


WB 


WB 


WB 


Asian 
Development 
Bank 

UNDP 


IFAD 


WB 


WB 


UNDP 


UNDP 


WB 


UNDP 


UNIDO 


UNDP 


WB 


China 


Brazil 


Bangladesh 


China 


Philippines 


Vietnam 


Ethiopia 


Regional, Cabo Verde, 
Liberia, Sierra Leone, 
Senegal 

Senegal 


Namibia 


India 


Mauritius 


Ukraine 


Uruguay 


Regional, Guinea, 
Liberia, Sierra Leone 


Index 


Note: Page numbers followed by b denotes material in a box; n, an endnote; italics, 


a figure; boldface, a table. 


accountability 40, 115, 145 

achievements 68-70, 69 

adaptability 109, 127 

additionality 44—46, 47b, 93-95, 129; 
defined 46, 95; developmental 46; 
dimensions 47; environmental 47b; 
financial 46; innovation 47b; institu- 
tional 47b; legal and regulatory 47b; 
socioeconomic 47b; utility 46—49 

adoption 85, 94, 109-110 

advisory panel 25 

Africa 50—52, 90n2, 96, 99, 100, 103 

agency evaluation units 16 

Amazon Regional Protected Areas 
Program (ARPA) model 86, 98, 100, 
108-111; ambition 98—99, 129; aware- 
ness 126 


Ba Be Protected Area 70, 71, 71b 

Bangladesh 106, 110, 111, 113 

behavioral change 94 

benefit boosting 115—121; clarity 115, 
118; credibility 115, 116; engagement 
115, 117-118; relevance 115, 116—117; 
responsiveness 116—117; timeliness 115, 
116 

biodiversity 3, 49, 53-54, 54, 55, 58-60, 
59, 72-74, 73, 76b, 87—88b, 123 

biosphere, loss of integrity 3 

book, organization 10—12 

Brazil 96, 98, 100, 106, 108—111 

budgeting 151; see also funding 


capacities 131, 153 
capacity-building 77, 111 
catalytic effects 102—103 


challenges 120 

change: depth and scale of 95; theory of 
80, 93-97, 94 

chemicals and waste 73, 78—80; health 
benefits and 126 

child projects 85, 89-90 

China 80, 86b, 96, 98, 100, 103, 106, 110 

China Renewable Energy Scale-Up 
Program (CRESP) 96, 98, 100, 103 

clarity 115, 118 

climate change 3, 5, 73, 75-78, 110, 122, 
123 

coherence 43b 

community health approach 126 

compensation 83 

competencies 153 

complex situations 136 

complexity 90, 91n8 

compromise 83 

conflict resolution commissioner 10 

context 122-123, 133b 

Convention on Biological Diversity 
(CBD) 13 

coordination units 16 

COREMAP 111 

corruption 125 

cost, incremental 45, 93; see also finance; 
funding 

cost-effectiveness 86, 109, 112—113 

Costa Rica 109, 111 

country considerations 130; single- 
country programs 85 

COVID-19 1, 3, 5, 6-7, 122-129, 133, 
136-137; wildlife trade and 124—125; 
see also Ebola crisis 


credibility 115, 134-135, 152 


182 Index 


CRESP see China Renewable Energy 
Scale-Up Program 
criteria 153 


Danube basin 97, 100 

decision-making, evidence-based 132b, 
133-134 

deforestation 28, 70, 71, 75b, 88—90 

demand, reducing 125 

demonstration 102 

design 66, 67, 68, 85, 108 

developmental additionality 46 

Director 163-164 

disease, infectious 3, 122—123, 124-125; 
see also COVID-19; Ebola crisis 

donors 98, 111 


East Godavari River Estuarine Ecosystem 
(EGREE) 76b 

Ebola crisis 126—129; see also COVID-19 

ecosystem: collapse 3; human health and 
122-123 

education see knowledge 

effectiveness 43b, 64, 115, 129, 153 

efficiency 43b, 74, 84, 133b, 153 

electricity see solar lamps 

engagement 115, 117—118 

environment: additionality 47b; benefits 
146-147; core components 3; COVID- 
19 2; damage and disasters 3; enabling 
77, 78; funding 123; global trends 3—5; 
multilateral agreements 78-79; social 
and economic development and 123 

Ethiopia 106, 112 

Europe see Danube basin 

evaluation 145—146; better 131-132, 
132-133b; comprehensive 42b, 149; 
country and country cluster 42b, 
149; defined 147-148; flowchart 147; 
follow-up 149-150; framework and 
recent innovations 43—49; GEF role 
13-20; impact 149; independent 134, 
137n1; methods 41—43; norms 151—159; 
ongoing 116-117; ownership 135; 
partnership 15—17; performance 148; 
planning 132—133b, 155; policy 115, 
142-168; principles 152—153; process 
149; program 148; project 148; purpose 
148; quality 118-120, 132; questions 
32-33; role 24; systems approach 
52-53; terminal 44, 62n1, 86, 155-156, 
157-158b, 158; thematic 42b, 149; 
timely 131-132; types 42b, 148-149; 


units 167—168; use 118-120, 135-136, 
148; value of 1 

evaluative function 163 

evidence 24, 150 


execution 129 


feedback 116, 117, 150 

fertilizer 3 

finance 36; additionality 46, 47b; budget- 
ing 151; climate change 4—5; environ- 
mental 4; GEF role 6—7; incremental 
cost 45, 93; management 37; OPS6 36; 
requirements 5; stability 98—99; see also 
funding 

fishing 127 

flexibility 112—113, 127 

focal areas 10, 105, 106—107, 113, 123; 
biodiversity 72-74, 73; chemicals and 
waste 73, 78—80; international waters 
73, 80-81; land degradation 73; mul- 
tifocal areas 73; performance 72—84; 
protected areas 74-75, 75b; single-focal 
area programs 85; strategy 81-82 

focal points 7, 16; operational 158b, 
167-168 

food security 28, 89—90, 126, 127 

forests see deforestation 

framework 42b, 43—49 

funding 123; climate change 77-78; 
replenishment 21-23, 22; see also 
finance 


GEF see Global Environment Facility 

gender equality 38, 71-72, 153 

geospatial analysis 53, 54, 55, 56, 137; 
efficiency measurement 56—60 

Global Cleantech Innovation Programme 
(GCIP) 46-49 

Global Commission on Adaptation 5 

global environment benefits (GEBs) 93, 
95; facility evaluations and studies 
173-175, 176—180; OPS6 35 

Global Environment Facility (GEF): 
Agency 10, 159, 161, 166—167; 
Assembly 8; constituencies 91n4; 
Conventions 8—9; Council 8, 16, 17, 
25, 140, 146, 159, 160-161, 161, 162; 
effectiveness 64—92; evaluation 13-14, 
18-19; funding 21; GEF-7 26-27, 116; 
institutional framework 7—10, 19n6; 
Instrument 146; Operational Focal 
Points (OFPs) 161; partnerships 8—10, 
72, 161; performance 17-19, 31, 64—92; 


portfolio 2, 84—85; projects 14, 16; 
relevance 28, 29-30, 29—30; replen- 
ishment 21—23, 22, 25—27, 116-117; 
role 6—7; Secretariat 9-10, 14, 16, 17, 
159, 160, 161, 165—166; sustainability 
64—92; Trustee 9; valuation architec- 
ture 15-17 

Global Environment Facility Evaluation 
Policy 140—169 

Global Risk Report 2020 3 

Global Wildlife Program 
87-88b 

GLOBALLAST 111 

good practices 150 

governance 36, 127, 130—131; additional- 
ity 47b; OPS6 36; structure 119 

government: agencies 111; federal and 
state 102; ownership 99 

guidance/guidelines 131, 147 

Guinea, Ebola 127 


housing, low-cost, quickly built 48b 
human-environment nexus 1—2 
human health see COVID-19; 
disease, infectious; Ebola 


IAP see Integrated Approach Pilot 

IEO see Independent Evaluation Office 

impact 42b, 43b, 96; additionality and 
95 

impartiality 152 

implementation 66—67, 67, 68, 71-72, 94, 
129; advocates 110—111; capacity, local 
institutions 99; national plan (NIP) 79; 
quality 99—100 

independence 134, 137n1, 152, 163 

Independent Evaluation Office (IEO) 
9, 15, 16, 40—41, 115, 160, 161, 162- 
165; evaluations 41—49; independence 
134; publications 41; replenishment 
and 24-26; reporting 115—120; role 
15-17 

India 75, 76b, 85, 96, 98, 99, 100, 110, 
111 

indicators 153; attributable 153; meas- 
urable 153; relevant 154; specific 153; 
timely 154; trackable 154 

indigenous peoples 72 

Indonesia 106 

industry 126 

innovation, additionality 47b 

institutions: additionality 47b; capacity 
94; learning 115; systems 130—131 
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Integrated Approach Pilot (IAP) 27—28, 
79, 84-85, 88—90, 116 

integration: focus 37; organizing principle 
28; programs 123 

integrity 153 

international cooperation 133 

international waters 73, 80—81 


knowledge/knowledge management 94, 
126, 133b, 136, 150-151 


land degradation 57—58, 62n5, 73, 81-82; 
neutrality 91n7 

land system change 3 

learning 40, 145, 150 

least developed countries (LDCs) 24; 
climate change fund 77-78 

legal and regulatory framework 101—102b; 
additionality 47b 

lessons learned 122-139, 150; applying 
150; context 122-123; COVID-19 
136-137; evaluations, better 131-132, 
132-133b; pandemic 124—129; part- 
nerships 130—131; political and global 
factors 133—136; sustainability of out- 
comes 130; systems approach to com- 
plex relationships 136; transformational 
change and scaling up 129-130 

Liberia, Ebola 127 

lighting. See solar lamps 

Lighting Africa 50—52, 99, 100, 103 

limitations 68-70, 69 

livelihoods 85, 125 

local agency 126 


M&E. See monitoring and evaluation 

Macedonia 107 

mainstreaming 75, 85, 86b, 100, 102; 
chemicals and waste projects 79—80 

Management Action Record (MAR) 118, 
149-150 

management response 149 

manufacturing 110; disease and 126 

market: change 98, 100; forces, harnessing 
129; transformation 52 

Mauritius 107, 112 

messages, audience 133b 

methodology; 

credibility 134; mixed 52-53, 55, 61 

Mexico 74, 75b, 106, 107 

minimum requirements/standards 72, 
155-159, 156-158b 

monitoring 145-146; flowchart 147 


184 Index 


monitoring and evaluation (M&E) 13, 
14-15, 86, 87—-88b; application 157b; 
design 66, 67, 68, 156b; levels and 
responsible agencies 160 

Montreal Protocol 79 

multifocal areas and programs 73, 82—84, 
85 

Multilateral Organizations 
Performance Assessment Network 


(MOPAN) 17, 116 


Namibia 96, 98, 99, 107, 109 

National Commission on Natural 
Protected Areas (CONANP) 75b 

National System of Natural Protected 
Areas (SINAP) 75b 

normative function 163 

North Africa 90n2 


OECD see Organisation for Economic 
Co-operation and Development 

official development assistance (ODA) 21, 
22, 24 

operational governance 38 

OPS see Overall Performance Study 

OPS6 23, 25-38, 132b; component 
evaluations and synthesis 27—31; 
conclusions 34—36; evaluation pro- 
cess 134—135; feedback 116; gender 
equality 72; recommendations 
37-38 

OPS7 23, 137; replenishment 117; theme 
1 28; theme 2 31 

organic pollutants, persistent (POPs) 
79-80 

Organisation for Economic Co-operation 
and Development (OECD) 5; DAC 
criteria 43b 

outcomes 45, 64—65, 65; additionality and 
95; influences on 84—86, 86b, 87—88b; 
sustainability 130 

Overall Performance Study (OPS) 17; 
Independent Advisory Panel Statement 
170-172; meetings 25—27, 117; OPS5 
18-19, 31 


Pakistan 48b 

pandemic 124—129; see also COVID-19; 
Ebola crisis 

Paris Agreement 4, 76—77 

participation 153 

partnerships 8, 15—17, 82, 119, 130-131, 
147; donor 98, 111 


Partnerships in Environmental 
Management for the Seas of East Asia 
(PEMSEA) 112, 113 

payment for ecosystem services (PES) 109, 
111 

performance 35—36, 42b, 90n1; interven- 
tions 64-71 

pest management 80 

Philippines 106, 107, 109 

phosphorous 3 

pilot programs 88—90; scaling up 108 

planetary boundaries 3 

planning, evaluation and 132—133b, 151 

policies 77, 98, 131 

policy-making, evidence-based 115 

political and global factors 133—136 

political will 125 

POPs see organic pollutants, persistent 

population growth 3 

prioritization 134 

private enterprises 100 

private sector 13, 37 

process evaluations 42b 

program evaluations 42b 

progress, tracking 146 

projects: performance heterogeneity by 
regions 65—66, 65; size 100, 130 

protected areas and area systems 74-75, 


75b, 76b 


quality, evaluations 131-132 
quality-at-entry review 78 


recommendations 37—38, 117 

regions: governance 113; networks 113; 
sustainability 68-70, 69; variation 
65—66, 65 

relevance 43b, 95, 115, 116-117, 153 

remote sensing 53 
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